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PREFACE 



To undertake the evaluation of five model language arts/ 
reading projects geographically dispersed and operationally unique 
provides a challenge to any group of evaluators. There had to be 
enough freedom from constraints that each center could develop into 
its own personality — yet some uniformity was needed to determine 
if the expenditure of one million dollars was effective. 

Needless to say, opportunities for failure in such an 
undertaking were prevalent. Many aspects of evaluation needed to 
be coordinated; yet opportunities for coordination were rare. 
However, this report is the results of our labor. From it we have 
learned much and have planned many ways to do a better job if the 
opportunity again arises. 

Overall, we feel that much was accomplished, that students 
and teachers learned a great deal. We are happy that the year and 
the opportunity existed? we are unhappy that the five projects were 
not given longer life, for one school year of operation is hardly 
enough to work out the kinks of such complex programs. 

We take full responsibility for what we have written without 
bragging or apologizing. We thank the many who have assisted us in 
this task, especially the five project directors and their staffs. 
Dr. Robert Hogan and Dr. Doris Gunderson. 

William G. Katzenmeyer 
Hugh I. Peck 
Robert A. Pitillo 
Richard S. Ray 
Durham, North Carolina 
November, 1971 
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INTRODUCTION 



In January 19*^0, representatives of the Bureau of 
Educational Personnel Development (BEPD) approached Robert Hogan, 
Executive Secretary, and other representatives of the National 
Coxincil of Teachers of English (NOTE) to determine if the organi- 
zation would cooperate in a joint endeavor with BEPD to plan five 
language arts/reading programs for inservice training, 

NOTE agreed, and the general goals of the project were 
established. Those objectives were as follows: 

1. The projects were to develop inservice training 
programs, non-traditional in nature, which could 
be transferred, in whole or in part, to other in- 
stitutions interested in upgrading training pro- 
grams in language arts/reading. 

2. The projects in their training prograiws were to 
emphasize the language base of reading, rather than 
stressing reading as an isolated skill area. 

3. The projects were to establish a close working 
relationship between a public school system and a 
university, so that the school system could benefit 
from the expertise of university personnel and to 
enable university teacher-trainers to more readily 
adopt new training methodology developed as a re- 
sult of the project. 
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4, The projects were to emphasize training of teachers 
and administrators in the primary grades* 

5, The projects were to place emphasis on the training 

needs of teachers and administrators in inner city schools. 

BEPD and NCTE did not offer a general competition on a 
national basis for receipt of a project grant. Rather, institutions 
were selected for funding which had demonstrated in the past some 
skill in developing innovative approaches to inservice training for 
teachers in language arts and reading. That, plus the need for 
geographic distribution and the desire for some diversity, were the 
controlling factors in selection of the five project sites. 

By mid-February/ 1970, the sites had been selected. The 

sites, institutional affiliation and the major project administrators 

at that time were: 

Berkeley, California - University of California - 
Dr, Robert Roddell 

Columbus, Ohio - Ohio State University - Dr, Charlotte Huck 

Chapel Hill, North Carolina - The Learning Institute of 
North Carolina - Mrs, Helen Wolff 

Philadelphia, Pennsylvania - Temple University - Dr, Howard Blake 
Portland, Oregon - Portland State University - Dr, William Jenkins 

As projects developed two changes were made regarding project 
directors, Dr, Colin Dunkeld became director of the Portland project 
and Dr, Paul Pritchard became director of the Chapel Hill project and 
principal of Seawell School, With the exception of the Chapel 
Hill, North Carolina project, each of the projects was to have a direct 
affiliation with a university through the project director, who was in a 
professional position at one of the named universities. The N,C. project was 
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to be administered by the Learning Institute of North Carolina, a 
non-profit research and development group in that state. Consul- 
tation and planning relationships were to be established with a 
number of teacher training institutions » 

In some instances, the institution was to be the funding 
agent; in others, the public school system, NOTE was to bear the 
major portion of responsibility for planning the project, arranging 
meetings of project representatives with BEPD and NOTE officials, 
for evaluation of the national project, and for dissemination of 
information about the project. The involvement of NOTE demonstrated 
one of the strategies of BEPD to achieve more rapid improvement in 
teacher training; that is, to utilize organizations of education 
professionals to stimulate innovation and reform in teacher training 
in order to upgrade the quality of educational offerings for children- 

NOTE called the first meeting of project representatives for 
March during the annual meeting of the American Education Research 
Association in Minneapolis. At that time the general guidelines for 
the project were discussed and adapted, funding arrangements were 
revealed, and project evaluation possibilities were discussed. 

It was decided that in view of the potential impact of this 
project on national legislation, ii particular the Right to Read 
program and the continuity of training funds through the Office of 
Education, a national evaluation of the five projects was necessary 
as well as internal project evaluations. Areas needing study for 
evaluative purposes were identified as follows: 

1) The impact of the training program on teachers 
competing in teaching language arts and reading; 
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2) Changes in teacher attitudes as they relate to 
language arts instruction as a result of the 
training programs; 

3) Observable changes in student language arts/ 
reading as a result of the training programs; and 

4) The cost/accountability of the projects in regard 
to their impact on student?, teachers, adminis- 
tration, school systems and institutions. 

A second meeting of project representatives was to be held 
in Anaheim, California, during the annual meeting of the Internat- 
ional Reading Association in early May, 1970. 

In the interim between the March and May meetings, NCTE 
asked representatives of the Learning Institute of North Carolina 
to design an evaluation plan for the five projects and to be pre- 
pared to discuss this plan in detail with project directors at the 
May meeting. Subsequently, Specialized Educational Consultant Ser- 
vices (SPECS) of Durham, North Carolina, developed the evaluation 
plan and entered into agreements with NCTE to conduct the national 
evaluation. UNC agreed to contribute its evaluation and dissemi- 
nation capabilities to the degree that they were desired and needed. 
The SPECS team was composed of the following persons, whose efforts 
were supplemented by data management specialists and clerical per- 
sonnel: 

Dr. William Katzenmeyer - Duke University 
Dr. Hugh Peck - LINC 
Dr. Robert Pitillo - Duke University 
Dr. Richard Ray - LINC 
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At the May meeting the specifics of the evaluation plan 
were discussed and project directors were informed of data which 
should be gathered. Plans for reporting this data to the evaluation 
team were made. 

Appendix A describes the SPECS evaluation plan agreed to 
by NCTE and BEPD. 

The national coordinated effort of NCTE and USOE was one 
of the first times that the U. S» Office of Education had cooperated 
with a national professional organization to implement specific 
programs that related to the organization's discipline. 
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CHAPTER II 
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It was the desire of both the National Council of Teachers 
of English and the S. Office of Education that the five projects 
under consideration be evaluated at two levels. First, each of the 
five projects should set aside funds for a local or project evalua- 
tion. Therefore, each project would be carefully evaluated as an 
entity in itself. Second, since each of the five was a part of a 
national program, some type of evaluation should be planned that 
would consider the value of these five basically different models 
in an overall way* Further, it was agreed that all projects would 
develop a basic data base and the national coordinated evaluation 
would use as much as possible the same data and analyze it across 
projects. 

The work of the national coordinated evaluation team was 
delineated as program evaluation, as opposed to research. Basically, 
the adoption of an evaluation model for this purpose negated the 
use of any control groups. It was agreed from the initial meeting 
by all represented (USOE, NCTE and project directors) that for the 
purposes of the national evaluation control group would not be in- 
volved. If local evaluators desired such, they were not forbidden, 
however, such data would not be reported to the coordinated evalua- 
tion team. 

Five major areas were selected as focuses for the national 
coordinated evaluation. Each of these might be spotlighted by use 
of a question* 
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1. What effect did the centers have on pupil performance? 

2. What effect did the centers have on teacher attitudes 
and practices in reading and language arts? 

3 . What individual programs or activities can be isolated 
and replicated at centers for broader national 
application? 

4. What model can be established that will provide oppor- 
tunities for changing teacher training practices and 
reaching more teachers? 

5. What efforts of the centers are effectively reaching 
more teachers? 

TO investigate the first of the five objective plans were 
made to collect pre- and post-test data on all children directly a 
part of the project schools . The national coordinated evaluation 
team provided a Student Data Card Format (flee Appendix B) which 
we hoped would provide some uniformity to data collection procedure. 
It was the goal, however, that the program of the center should come 
first and the evaluation data collection should not determine program 
content. We had hoped that conclusions regarding student 
performance could be made based on data collected. 

Two major efforts were undertaken to look at the "teacher 
variable." A Teacher Data Card Format was suggested by the national 
coordinated evaluation team (see Appendix B) and a questionnaire was 
prepared to survey teacher practices and attitudes. The SPECS 
Teacher Practices and Attitudes Survey is included in Appendix C. 

In summary, in ofder to determine if participation in the 
project had an effect on teacher attitude toward or practices in 
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the language arts/reading areas, a Teacher Practices and Attitudes 
Survey was administered in a pre-test/pbst-test design. Results of 
these studies are presented in Chapter V. 

The same evaluation strategy was used to study the effect 
of the projects on students' performance. It should be remembered 
that each project adopted and administered its own testing program, 
thus, there was no uniform program throughout. The national coordi- 
nated evaluation, therefore, attempted to use the existing data to 
study performance across centers. 

To determine if oth«ir objectives were met the evaluation 
team relied heavily on site visits, both formal and in connection 
with other meetings, on conversation with the various project staff, 
and on formal reports from the project directors to the evaluation 
team. Each project site was visited by at least two members of the 
evaluation team. One team member visited all five sites at least 
once and was in most project schools. To many, site visitation may 
seem among the least objective and least desirable methods of evalua- 
tion. If they are used as an only technique, we would agree. When 
site visits are combined with other information they make the total 
evaluation more real and at least allow the evaluation team to put 
the various aspects of the evaluation into a similar framework. 

Project directors were asked to report to the evaluation 
team by following a specific set of questions (see Appendix d) as 
well as to respond to certain open-end questions. These reports 
were amazingly candid and useful in preparing this document. The 
evaluation team received complete cooperation from project staff 
and great deal of trust in our work as we searched for effective, 
but non-interf erring evaluation methods. 
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One phase of the evaluation really never became fruitful. 
We had hoped to provide a good deal of data regarding cost analysis 
and effectiveness » For many reasons this never was completed. 
First, each project was on a different fiscal year: one opening in 
April, 1970 (the first to open) and the last one closing down its 
fiscal year in December, 1971. Each project had a specific fiscal 
officer, to whom they were responsible — each officer had a diffe- 
rent system of accounting and cross comparisons were again meaninc- 
less. Finally, project directors were not Planning, Programming 
and Budgeting Systems experts and were rightly much more interested 
in program content than money matters. 

In evaluation design, strategy, implementation and outcome 
there are fundamental changes that must be made when original plans 
do not pan out and for any number of reasons "best laid plans" oft 
go astray. For example, we had planned on a pre/post*^assessment 
of pupil performance in all five centers. One center, because of 
the schedule of the local testing program, was unable to comply and 
used a post-test-only approach. We have appended to this report 
the original evaluation strategy. Parts of it were carried to im- 
plementation. Parts fell to the wayside. 

Among the many lessons learned from the implementation of 
a coordinated evaluation for diverse projects (geographically and 
programmatically) one major lesson seems clear. Close liaison is 
necessary, some uniform agreement on instrumentation, at least a 
minimum agreed upon amount, must be implemented. If both loc^l 
and coordinated evaluations are planned, all these evaluators must 
meet early and often to bring such a project to a fruitful end. 
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The struggle for balance between the individual program and the 
coordinated evaluation is not an easy one, but it should be hammered 
out early under the watchful eyes of the funding agencies. 
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This chapter reports the results of the five site visits 
made by meinbers of the SPECS evaluation team. Reports v/ere written 
by various members of the evaluation team, and this chapter reports 
those documents directly. Since each project is unique and since 
all staff members did not visit all centers, the visitation reports 
do not follov/ a uniform format. Further, conclusion based on the 
site visits will be made on a site-by-site basis, rather than 
across sites. 
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Site-Visit Report on: Project DELTA - Berkeley, California 

Washington Elementary School 
Dates of visit: April 27 and 28, 1971 

May 5 and 6, 1971 

Personnel on Visit: Dr. Richard S. Ray 

Dr. Hugh I, Peck 

Dr. Robert A. Pitillo 
Dr . Hugh I , Peck 

Project DELTA was a cooperative program in many ways. Spon- 
sored by a joint effort of the U. S, Office of Education and the 
National Council of Teachers of English, it was cooperatively oper- 
ated between the University of California/Berkeley ^rid Berkeley Public 
Schools , specif ically Washington Elementary School. DELTA constituted 
the 600 students in Washington School, the school staff of 24 person- 
nel plus three full-time, five part-time and one project director? 
the later three categories were financed from DELTA funds. DELTA 
was housed in Washington Elementary School and focused its major 
attention on the faculty and students of that school. 

DELTA organized itself in a functional way, choosing to 
call each of its major efforts strands . These strands involved 
five carefully focused program components, each chosen for its re- 
lationship to a complete language arts/reading program. The five 
strands were: 

Oral and Written Language Development 

Literature and Self-Concept 

Comprehension (Critical Thinking and Questioning) 
Decoding (Early Word Attack Instruction) 
Parent Participation 
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Although it was never seen as a strand, the development of new and 
innovative measures in the language arts/reading areas was a major 
focus and one of the major strengths of Project DELTA. 

One member of the DELTA staff took the responsibility for 
developing each of the five strands • In many ways each strand be- 
came the private concern of that staff member. Teachers within the 
Washington School faculty became strand members and rotation systems 
were worked out to assure that faculty members were involved in a 
number of strards* Each strand working cooperatively with the DELTA 
staff member and their teacher group set objectives for their strand 
meetings. Usually these objectives were stated in written form and 
progress toward these objectives was charted through minutes of 
strand meetings. 

During the site visit the evaluation team attended a meeting 
of the parent involvement strand, therefore, having the opportunity 
to actually see the strand in operation and observe the interaction 
between DELTA staff, members of Washington School faculty and local 
parent representatives. The visitation team had opportunities to 
meet with each strand leader cind in some cases members of the Wash- 
ington School faculty who were strand participsmts . During these 
meetings, goals of each strand were explained and activities that 
each strand undertook were illustrated. Also, during these meetings 
we were able to meet and gain a broader knowledge of the research 
aspects of Project DELTA. 

The literature and self-concept strand had as its focus 
the use of stories, poems or other literary works as a starting 
point in the improvement of self -concept. Children us^d characters 
in stories to develop ideas about how people feel about themselves 
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and others. These people became models or identifiers for the 
children fro use in the process of building a more positive self- 
concept. Further, these stories provided a take -off point that 
students could use in writing their own stories and reflecting 
through these stories their self-assessments. In the opinion of 
the visiting team, this strand had developed further the concept of 
using literature, modern and classic, to assist children in under- 
standing themselves and their environment than any similar program 
of which we were aware. 

Decoding skills are those necessary for the beginning stu- 
dent to learn word attack abilities. The DELTA decoding strand 
emphasized both decoding strategies and decoding for contentj 
specifically they taught the structural elements of the words and 
the derivation of the content of the word from an understanding of 
the elements. The approach used to develop these decoding skills, 
though rather elaborate, seemed to be operating well, and teachers 
who had been involved in the strand training seemed to be effectively 
applying the decoding plan. 

Perhaps the leader or wrap-up strand was that of oral and 
written expression. The rationale seemed to be that if language 
arts and reading are effectively taught, both the oral and written 
expression of the student will be positively changed. Thus, this 
strand not only depended in large part on the effectiveness of other 
strands, but built heavily on the others to expand the language 
horizons of the students. The Oral Language Inventory, a part of 
this strand, indicated six specific activities that made up the 
strand: quality of thought, organization of expression, quality 
^and control of language, fluency of language, personal response to 
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language and technical skills in oral expression. There seemed to 
be a great deal of emphasis on non-standard dialects and how they 
were a part of the total life of the child. 

The DELTA comprehension strand should not be thought of in 
the routine sense of the meaning applied to comprehension, especially 
a subscale of a test called reading "comprehension." DELTA'S use 
of the term is applied to an entire taxonomy for classifying the 
interaction in classroom discussion and reaction. In many ways the 
term applies to the teachers' own comprehension of themselves in 
action. DELTA has developed a unique taxonomy for the analysis of 
classroom interaction. The taxonomy looks at four aspects of the 
interaction: (1) who was speaking (teacher or child), (2) what 
type of verbalization went on (comment, question or response), (3) 
what level of comprehension was involved in the interaction Cf actual, 
interpretive or applicative) , and finally, (4) what role did the 
teacher play or what strategy did he or she follow '.focusing, ig- 
noring, controlling, receiving, clarifying, extending or raising). 
As the site visitation team viewed this strand, we felt that the 
process of the taxonomy was operating more effectively than its 
purpose. That is, the teacher and strand leader seemed unclear as 
to some goal for the taxonomy — some reason for its use some- 
where such action should take the teacher. If such questions were 
made clearer to the users, the taxonomy would increase in value. 

As an overview of our visit to Project DELTA, wc would like 
to point out the following areas for improvement, should the project 
continue or should others wish to adopt the DELTA model. 

First, the DELTA strand needed a great deal of weaving 
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together. We were impressed with the strand approach, however, 
much could have been done to bring together the five strands into 
a total language arts/reading program. 

Second, the decision makers within DELTA were in hopes that 
a second year would provide them with their outreach opportunity — 
this was expected by all» As it turned out, there was no second 
year and there was a good deal of hurried planning to see that there 
would be reasonable residual effect of the projects If there was a 
next time around, we cannot plan for that second year • 

Finally, it seemed to the visitation team that DELTA never 
became an integral part of the local school. Perhaps earlier in 
the history of the program more groundwork could be laid to prepare 
the school, the university and the community for the institution of 
such a project. 

We hasten to add that we saw many positive things about the 
project. Of all the projects involved, DELTA had a greater research 
focus than the others, diie, we are sure, to the leadership of the 
prdject director. Evaluation and the development of new instruments 
had a greater focus and these were well planned and developed ideas. 
DELTA, it seems to us, really tackled the guts of language arts/ 
reading problems, attempting to delineate the problems into some 
reasonable parts and find innovative ways to search for usable and 
replicable solutions. 
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Site Visit Report on: Seawell Elementary School 

Chapel Hill, North Carolina 
Evaluators on Visit: Dr. Hugh I. Peck 

Dr. Richard S* Ray 

Seawell Elementary School is located in Chapel Hill, North 
Carolina, home of the University of North Carolina* It opened in 
the fall of 1970 with a "pod" design for multi-aged/teara teaching 
under the direction of Dr. Paul Pritchard. At its opening it was 
one of the NCTE/USOE model training projects for language arts and 
reading. Though the architecture of the school was planned separately 
from the NCTE/USOE project, the two were compatible and in many ways 
seemed made for each other. Teachers employed at Seawell had volun- 
teered in the spring of 1970 without realizing that Seawell would 
become a major training component. 

Seawell School and the Learning Institute of North Carolina 
teamed with the NCTE/USOE groups to form a training or outreach 
school that would provide through planned rotation training for 
teachers in innovative techniques for language arts and reading in- 
struction. Satellite schools from nearby school districts were 
selected to cooperate in the training sequence. There were eight 
satellite schools located in seven school districts in North Carolina. 

A sequence of events was outlined as the training component 
of Seawell. Administrators of participating school units made a 
site visit to orient them to the school, its philosophy, program 
and facilities. Before teachers visited Seawell i a field services 
person from LINC/Seawell would visit the school to assist in orien- 
tation and planning for the satellite school visits to Seawell. 
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Step three was a four-day visit by teachers from the satellite 
schools to the Seawell School. Six weeks after the visit to 
Seawell, the field services person would again visit the satellite 
school as follow up and to provide whatever services he could in 
assisting the school to implement as much of the Seawell program as 
the individual teacher wanted to undertake • 

Two types of evaluation were planned: 1) a series of 
instruments to determine the effect that being involved in the 
project had on the attitudes and practices of participating teachers, 
and 2) an assessment of the test performance of Seawell students 
(1-6) on a pre/post design* 

Project evaluators made an on-^site visit during the yisita-* 
tion period to one of the satellite schools. Further^ project 
evaluators were able to talk with members of the Seawell staff 
from time to time during the year. We felt the plan for inservice 
education was well organized and had a good cheince of being the most 
effective school in the "outreach" or "multipliers" effect of the 
language arts/reading models. On the whole, teachers participating 
from satellite schools felt the experience was worthwhile and were 
returning with ideas for implementation in their classrooms* 

We would point out the following as "food for thought" as 
others look toward the adoption and adaptation of this model. 

Teachers who are the "on staff" group have dual responsi- 
bilities teaching their children and training and having satellite 
teachers. These will in many cases conflict^ especially if the 
staff teacher sees her job as keeping the visiting teacher "busy." 
The most effective approach seemed to be where "pod" teachers con- 
tinued their teaching and learning process, and visiting teachers 
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entered into the serving of what was happening and worked directly 
with the children. ^ 

Visitation to Seawell was divided between class time for 
visiting teachers usually directed by LINC personnel, afternoon 
sessions on creative teaching, and in pod time for observation and 
participation. However, some schools that were visiting had been 
operating similar programs for longer periods of time. Perhaps 
more careful selection of satellite schools is called for^ 

Teachers in Seawell need time and release time. They must 
plcin for children and for other teachers. They should know what 
is expected of them and be compensated for the additional responsi- 
bility they accept. It will take a strong and dedicated group of 
teachers and administrators to operate a school simultaneously with 
an inservice training program. The writer visited all five of the 
NCTE/USOE language arts/reading models. This model was providing 
more outreach, doing more inservice training and reaching a greater 
number of practicing teachers than any of the others. 
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Site-Visit Report on: In-Service Program in Reading/Language Arts, 

Columbus , Ohio 

Kent and Indianola Schools 

Date of Visitt June 3, 1971 

Personnel on Visit: Dr, Winiam G. Katzenmeyer 

Dr. Hugh I. Peck 
Dr* Robert A* Pitillo 
Dr. Richard S. Ray 

The four members of the evaluation team visited each of the 
two schools^ Kent and indianola, involved in the Ohio State Univer- 
sity/Golumbus Public Schools Language Arts Project on June 3, 1971* 
In addition, the project director and teaching associates conducted 
a briefing for the evaluation team at the end of the day. 

The purpose of the visit by the evaluation team was to see 
the program in action and to get the "feel" of what was taking place. 
The evaluation team made no attempt to assess the achievement of 
students based on the visit. 

One of the principle objectives of the new approach was to 
get the program of instruction on an individual basis with perfor- 
mance level instruction. One of the vehicles employed to effect 
these objectives was the open classroom/interest-center approach. 

Members of the evaluation team visited each of the class- 
rooms in both schools. It was obvious that instruction was interest 
center based. Youngsters were active in classrooms working in read- 
ing groups, arithmetic groups, etc-, without teacher domination. 
One striking example of the interest center approach being applied 
as a vehicle for skills development was the use of the hot-plates 
for preparation of food. Children had prepared everything from 
fudge, which did not "turn out," to potato salad, which was good. 
O :h concepts and reading skills were essential to the success of 
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the cooking projects. Thus, each youngster was motivated to read 
the recipe and compute the necessary ingredients. 

The visiting team had the opportunity to talk informally 
with a number of teachers. Their general reaction to the project 
was supportive. Many teachers, some of whom admitted strong mis- 
givings about the project at the outset, stated that they could 
hot return to the traditional approach to teaching. In addition/ 
teachers reported that youngsters were happy and that the traditional 
"late year" discipline problems were almost non-existent. 

We were impressed by the explanation of one teacher who 
told of going immediately to an open classroom situation following 
the summer in-service training. Then, finding that she could not 
handle the new situation, she returned to a traditional environment 
— only to discover that she really preferred to be back in an open 
environment. As a result, she began a gradual transition to interest 
centers and open environments. At the time of our visit in the 
spring, her classrooms were operating as open environments. 

The project directors had reported to the evaluation team 
that one faculty had voted to disassociate itself from the project 
for the year 1971-72; A combination of events and circumstances 
led to this decision. Firt, one of the principals appeared to be 
less than enthusiastic about the program, and, second, misunder- 
standings resulted from contact with the university people. These 
misunderstandings appeared to have resulted from too much pressure 
for immediate change as well as the old communication problem. In 
spite of the "problem" changes were tcOcing place, teachers were 
committed to the new approach, and the program was entrenched. 
Q The briefing session with university people was well planned 
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infomation and denionstrated a high level of competence and leader- 
ship. The teaching associates knew their role^ displayed insight 
into the problems and appeared to be articulate people who will 
spread the program to other areas of the country. 

In summary, the evaluation team was favorably impressed 
with the program. In classrooms visited, teachers were using the 
interest center, child-centered approach to instruction. They 
were working closely with teacher associates and the concept of 
the open classroom appeared firmly entrenched. The reading program 
was relevant, functional, and the children were enthusiastic and 
happy. 
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Site-Visit Report on: Project PIRLT - Philadelphia, Pennsylvania 

Carver - Washington Schools 

Dates of Visit: March, 1971 

Personnel on Visit: Dr, William Katzenmeyer 

Dr* Robert A» Pittillo, Jr, 

The evaluation team visited the two schools in the 
Philadelphia project for the purpobe of viewing the classrooms i 
talking to teachers and project staff, and getting a "feel" for the 
program in action. During the two days the team spent in Philadelphia 
most of the classrooms involved in the project were observed. The 
team made no attempt to assess the achievement of students? however, 
special attention was given to instructional strategies and 
teacher involvement, commitment and reaction to the project. 

Before entering classrooms the team was briefed by project 
staff and the school principals. Examples of materials developed 
and progress to date were discussed. One member of the project 
staff spent a good portion of the first morning acquainting the 
team with objectives by teacher and project accomplishments. 

A teacher strike in Philadelphia which occurred early in 
the school year coupled with a high sensitivity of the staff to 
school- community faculty relationship resulted in a very cautious 
approach to the implementation of the program. 

During the first briefing of the team it became obvious 
that the project staff was knowledgeable about the program objectives 
and each teacher involved. The University people were working very 
closely with the administration and faculty of each school. 
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Thay were able to assess each classroom and teacher involved and 

an excellent relationship between university people and school staff 

was evident. 

Creative materials had been prepared to acquaint parents 
with the progreim. Moreover, teachers were being assisted with 
acquisition and development of instructional materials. 

Although it was difficult to identify the major changes 
taking place in classroom organizations it was apparent that teachers 
were moving to a child centered approach. Some classrooms had 
moved in the direction of interest centers, individual projects, 
and experience centered activities. Parent aides and student 
teachers were working with participating teachers. 

The evaluation team had the opportunity to talk at length 
with some of the participating teachers and to talk briefly with 
others. Teacher reaction to the project was very supportive. 
Project teachers stated that the program was providing a vehicle 
through which they could operationalize effective diagnostic techniques 
and productive teaching strategies. Participating teachers were 
enthusiastic about the future of the program, and they expressed 
the feeling that they were working together with strong support 
from university people and the principals. 

An interview with the principals revealed their support 
of the program and their desire to continue to work with the 
university staff. Continuing inservice training sessions were 
effective in that teachers could discuss real classroom problems 
with the university staff. The principals and teachers reported 



ERIC 



- 28 - 



that the project had a salutary effect on moral generating confidence 
and enthusiasm on the part of the participating teachers. 

In summary, classrooms were made attractive and alive by 
a variety of materials and activities. University personnel 
including graduate students / undergraduates, and professors were 
working as a team with the participating teachers and the principals. 
Progress was not as rapid as the project directors had planned; 
however, many difficulties were being overcome by time and 
determination. Teachers and the principals were confident that 
the program was sound and that they were making deliberate progress 
toward the goals. 



ERLC 



- 29 - 

Site-Visit Report ons Portland State University/Portland City 

Schools 

Date of Visits April 29, 1971 

Personnel on Visits Dr, Hugh I. Peck 

Dr. Richard S* Ray 

The Portland Project is a cooperative effort between Portland 
State University and the Portland City Schools. A number of teachers 
in Portland City Schools enrolled in this cooperative project in the 
language arts/reading area. Enrollment in the project meant that the 
enrollee would attend on-campus classes one day each week as well as 
receive support services from members of the project staff. This 
support was provided by two members of the staff who acted as "super- 
visors" for the project. 

During the site visit, project participants were on campus 
at Portland State University and the team spent the day with the 
teachers. Because it was an on-campus day, we were unable to visit 
the project schools. The morning of the site visit was spent at a 
lecture by Dr. Jenkins on the role of the library in the elementary 
school. Also, there was a meeting of the parents participating in 
the project. This meeting was held in an adjacent room and one 
member of the visitation team sat with the parents group. 

Members of the visitation team spent lunch with three teacher 
participants and had an oppportunity for informal conversations with 
them. One part of the visit was spent at the Northwest Regional 
Education Laboratory discussing the evaluation strategy. One 
unique aspects of the formative evaluation was the inclusion in 
the group of participants of a participant-observer who sat in with 
group as a member and as a critique observer. The function 
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of this person was to provide feedback to the staff personnel 
who has operated the program for that day. 

The parent participation section of this project was 
directed by a dynamic leader who was able to establish very active 
rapport with parents who were acting as aides and volunteers to other 
teachers throughout the system some of whom were in the project. 
As an observer the author was pleased with the manner in which the 
parent section operated and with the valid way in which parents 
took part in the program. Certainly this phase would need to be 
considered one of the highlights of our visit and one of the best 
examples of parent involvement in school functioning that we have 
seen « 

Reaction of teachers to the project was, as would be 
expected^ varied. While some teachers saw this as opportunity to 
gain some college credits most teachers felt that the project 
definately had lots of innovative techniques to offer them and 
that they would take advantage of these opportunities to improve 
their instructional program. There was a good deal of discussion 
concerning ways in which the supervisory personnel were using their 
time. Some teachers want more opportunity to work with these 
personnel/ other stated that they felt the supervisory personnel 
needed to be used only when teachers were having difficulty in 
their instructional program. 

The afternoon demonstration conducted by the project staff 
was one of the most innovative demonstrations on the use of dramatics 
or acting out stories to teach communications concepts that we had 
ever witnessed. Using the story of Peter Cottontail as a base, par- 
^'^''^ pants were instructed by the demonstration in immeasurable ways 
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in which dramatic plays could be used to teach both concepts and 
skills. Following the demonstration, an audio-visual demonstration 
on rabbits was presented. The film and the story fused into a major 
demonstration of the use of visual and dramatic effects to implement 
creative learning. It was easy to see how such approaches to teach- 
ing could really "turn-on" young students to education. 

As we review our site visitation to Portland, we regret that 
we were unable to visit schools participating in the program, but 
realize we could only get a scattered picture of the various aspects 
of the project. As is true with any project, Portland seemed to 
have some difficulty becoming an integral part of the school unit 
rather than an outside project. However, project decision makers 
were aware of this and steps were being taken to change this. The 
project was faced with the problem of how to select participants 
or if there would be opportunity to make such choices. 

We felt the major strengths of the project were the on-campus 
section and the innovative types of instruction that were provided 
there. Certainly, the project staff represented a team of dedicated 
and able educators. The two coordinating supervisors were extremely 
able and doing a very creditable job and helping teachers do a better 
job. Finally, we should mention the role that the project director 
played and the ability and enthusiasm that he had for the job he 
was doing. The Dean of the School of Education at Portland State 
University was the first project director and has given the project 
his unqualified support. 

We cannot close without stating again that we felt the 
parent involvement section was a real strength. As of this point 
in time, Portland is the only project that is assured of continued 
gj^(-iding and we are pleased that at least this one will continue. 
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A Closing Statement Regarding Site Visits 

We have stated earlier that we believed the site visit 
technique is of limited value • However, we further feel that 
without such site visits we would not tie together any reasonable 
type of evaluation. There is no substitute for being on-site at 
a project in order to get a veritable feel for the projects. We 
feel fortunate to have had the opportunity to visit each project • 
We need to repeat that all projects showed strengths, that each pro- 
ject developed a personality that in many ways reflected the person- 
ality of its director and staff. No project ever reaches Utopian 
proportions or operates without problems. Thfe team of project 
directors was one of the most capable, varied and yet uniform group 
with which we have worked. They were varied in their approaches to 
the language arts/reading field; they were uniform in their dedication 
to providing better teaching in our schools. 
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CHAPTER IV 
SUMMARY OF PROJECT DIRECTORS' REPORTS 



When the project directors met in Berkeley on May 3, 1971 ^ 
it was decided that each director would file a report to be included 
in the coordinated evaluation. A format developed by the project 
directors was prepared by SPECS and mailed to each center. Three 
of the directors completed the report. This report was designed to 
identify the impact of the project on: (1) the university ccrrarronityj 
(2) the public education community (teachers, administrators , super- 
visors from non-participating classroom schools and school systems) ; 
and (3) the parent community of the participating schools. The 
project directors' questionnaire is presented in Appendix D. 

Project Director's Report for the Ohio S tate University/Columbus 
Public Schools Project — Dr. Charlotte S. Huck, Project Director. 

The university community was involved on various levels. 
Nine professors filled the following project positions: 

2 - Directors 

1 - Supervisor of undergraduate student aides 
1 - Evaluator and consultant 

4 - Consultants (taught one or more seminar sessions) 
1 - Adjunct professor (worked half-time with the project 
as a team coordinator in one of the schools. 

Also participating were 8 graduate students/ 7 of whom assisted in 

planning and teaching the weekly seminars / working in direct contact 

with teachers in the schools. The eighth graduate student served as 

administrative assistants on a part-time assignment. Working as 

teachers' aides^ 68 undergraduate students assisted for periods of 

tw) full days per week for one school quarter. 
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The number, positions and geographic areas of visitors to 
the OSU/CPS Project schools are presented in Table 1* Two hundred 
and four visitors to the project were logged. They represented a 
variety of professional activities, both within and outside the 
field of education, and were predominantly from the state of Ohio. 

The number of formal presentations of the OSU/CPS Language 
Arts Project to various educational groups is listed in Table 2. 
Seven hundred and eighty-three presentations were recorded to in- 
terested persons, primarily public school personnel. 

Table 3 lists the informal presentations of the OSU/CPS 
Language Arts Project to non-participants by the project staff 
members. 

The percentages of visits by project personnel of Kent and 
Indianola Schools to other schools in the Columbus area are pre- 
sented in Table 4 • 

Percentages of project personnel who took one or more 
professional trips funded by the project during the school year are 
shown in Table 5, In Table 6 there is a listing of the trips made 
by project personnel to participate in NOTE project conference and 
professional meetings held in various cities. 

In addition to the formal presentations of Tables 1-6, the 
project directors were invited to discuss informal evidence of 
multiplier effect. The following section titled "Other Evidence 
of the 'Multiplier' Effect" is a narrative representing the response 
of the OSU/CPS project director to this invitation. 



ERLC 



TABLE 1 

NUMBER OF VISITS TO THE OHIO STATE UNIVERSITY/ 
COLUMBUS PUBLIC SCHOOLS PROJECT SCHOOLS 



Geographical Area Represented 



Position of Visitors 


Columbus 
Publ ic 
School s 


Columbus 
Area 


Ohio 


Out of 
State 


Unknown 


Total 


School Administrators 


13 


7 




0 


1 


2 


23 


Teachers 


31 


19 




11 


0 


0 


61 


University Personnel 




OSU/Other 










a) Professors & Instructors 

b) Unknown 




3 
1 


0 


1 


2 




9 
2 


College Students 

a) Undergraduates 

b) Graduate 

c) Rank unknown 




23 
8 




lU 
1» 






10 
23 
12 


Executives in 

Professional Organizations 








2 






2 


U.S. Office of Education 










2 




2 


Member School Advisory Board 




2 










2 


Parent 

a) Study Group 

b) From Other School 

c) Potential School Patrons 

d) General 


5 
13 

9 
2k 










2 


5 

13 
9 
26 


Reporter - T.V. 




\ 










1 


Posit ion Unknown 




2 








2 


i» 



TOTAL 95 66 32 5 6 20'» 
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KO«HAL PRESENTATIONS OF FHf OSU/CPS 










Approx. Number 


Columbus Board of Education 


II 


Laura Zirbes' Conference, OSU 




Colufnbus PubMc Schools Reading feochers 


7'j 


North End Organization for School Improvement 


3> 


Curriculum Class, Copital Un 1 versi ty. Col umbub , Ohio 


'iO 


CurrJwulum Committee - Columbus Board of Education 


1? 


Curriculum Committee - Columbus Public Elementary School 


\b 


Arlington Public School Teachers 


100 


First Community Church, Arlington, Ohio 


60 


P.T-A* Study G roUD - Indian ^nrinn*; <irhnnl 


20 


H.C.f.E. Elementary Conference - Panel 


250 


Los Angeles, California - Grou|) Meeting 


hO 


Ohio f<or<hern University to I.R.A. Group 


65 


rOFAL 


703 



Presentation of the Project to personnel from other EOPA/NCfE Projects. 

a) Approximately twenty teachers in tlie Philadelphia Projecl 
attended the presentation at the Project Directors' Heeling 
in Phi ladelphia. 

b) Five persons from the Portland and Chapel Hill Projects 
viewed the slides and discussed the OSU/CPS Project with 
tv/o Teaching Associates from the Ohio State University 
informally at the NCrt: Elementary Conference in 

Los Angeles, Colifornia. 
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INFORHAL PR;-:SENTAriONS OF THE OSU/CPS LANGUAGE ARTS PROJECT 
rO OTHERS BY PROJECT SfAFF MEMBERS 



Teaching School 
Associates Personnel 



School Administrators 




280 


Teachers 


52 


7'f 


School Personnel 


35 




University Personnel 


3 


6 


Col lege Students 


101 


25 


Cotnniunity Service People 




2 


Uews Media 




1 


Parents 


^7 


66 


Coniuiunity Leaders 


13 


3 


Un i dent i f i ed 




105 
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TABLE 4 

VISITS BY PROJECT PERSONNEL TO OTHER SCHOOLS 



PARTICIPATING SCHOOLS 
Indianola Kent Total 





Number 


Percent 


Number 


Percent 


Number 


Percent 


Classroom 
Teachers 


5 


62% 


9 


60% 


14 


61% 


Special 
Teachers* 


4 


80% 


3 


50%** 


5 


62% 


Principals 


0 


0% 


1 


100% 


1 


50% 


Total 


9 


61% 


13 


54% 


20 


61%*** 



*A11 participants were given the opportunity to take a professional 
trip. For various reasons some elected not to take one. 

♦♦Participants were given release time to make visits • 

***Three special teachers (Art, Music and Physical Education) served 
both project schools. They were included in the breakdown for both 
schools, accounting for the difference in the sum of the parts re- 
garding special teachers* 
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TABLE 5 

PROJECT PERSONNEL WHO TOOK ONE OR MORE PROFESSIONAL 
TRIPS FUNDED BY THE PROJECT 



PARTICIPATING SCHOOLS 
Indianola Kent Total 





Number 


Percent 


Number 


Percent ' 


Number 


Percent 


Classroom 
Teachers 


7 


87%* 


9 


60%** 


16 


70%** 


Special 
Teachers 


1 


20%*** 


3 


50%*** 




50% 


Principals 


1 


100% 


1 


100% 


2 


100% 


Total*** 


9 


60% 


13 


60% 


22 


66% 



*The only classroom teacher at Indianola School who did not make a 
trip was scheduled to go but was hindered due to illness, 

**A11 participants were given the opportunity to take a professional 
trip* For various reasons some elected not to take one* 



***Three special teachers {Art/ Music and Physical Education) served 
both project schools • They were included in the breakdown for both 
schools, accounting for the difference in thi^ sum of the parts re^ 
garding special teachers . 

NOTE: In lieu of the professional trip, one special teacher (music) 
participated in a music workshop at Capital University. This 
was not counted as a trip, however, in this tally* 

NOTE: One classroom teacher dropped out of the project after having 

taken a trip* This trip was included in this tally, accounting 
for the variance in total number of teachers on the table and 
on the current roster of participants ♦ 
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TABLE 6 

PROFESSIONAL TRIPS HAOE RY OSU/CPS 
LANGUAGE ARTS PROJECT PERSONNEL 



Directors T.A.'s Project Teachers Total 

School 

.„,Meettngs - ■ "Principals 



Quail Roost Conference 

Durham^Rdleighi North Carolina 
o 


2 




1 


2 




— — — 


N,C,T.E, National Convention 
Atlanta, Georgia 


2 








2 


k 


Philadelphia/Temple Project 

Philadelphia, Pennsylvania 


2 






2 


6 


10 


Martin luther King School 
Evans ton, tUinols 


2 




3 


2 


8 


15 


N«C«T«E. Conference in Los Angeles 
and Project Directors Meeting 
In San Francisco, Callfornfj 


2 




3 






5 


International Reading Association 
National Con ven t { on 
Atlantic City, New Jersey 










2 


2 


Childhood Education Association 
International Conference 
Milwaukee, Wisconsin 










1 


1 


Seawel 1 School 

Chapel W\\\, North Carolina 






I 




2 


3 
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OTHER EVIDENCE OP "MULTIPLIER" EFFECT 

1 * Three teachers from Ar 1 i ng ton School s ( Wi 1 son Hill Els^mentar y ) 
heard about the program by way of a student teacher who is the 
advisee of the husband of one of the Teaching Associates in the 
Project. Being interested^ the teachers visited the Project 
Schools/ went back and rearranged their own classrooms, setting 
up learning centers in them. 

2. A video tape demonstrating a Language Experience lesson was 
developed in the Project by two Teaching Associates. This tape 
has been used as follows: 

a. In training approximately 30 Parent Volunteers, a cooperative 
effort involving the Reading Center at the Ohio State University, 
Columbus Public Schools, the Urban Education Coalition 

Right- to-Read Project and Parent Volunteers. 

b. In undergraduate reading methods courses at the Ohio State 
University. 

3. Five Teaching Associates in the Project have also taught 
undergraduate reading, language arts or children's literature 
courses for one or more quarters during this year. 

4. Four Teaching Associates in the Project have assisted with the 
training of teachers in various other federally funded in-service 
education projects throughout the state of Ohio, involving twenty 
teachers and administrators. A third Teaching Associate conducted 
a one-day workshop at Union City, Ohio while a fourth conducted 
three sessions at Highland Park School in the Southwest School 
District. 
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5. Data for two doctoral research studies are being gathered in one 
Project School. One investigator is comparing the amount of 

student initiated activities in an informal classroom with t 

in a traditional classroom. The second investigator is looking 

at the relationship between children's creativity and their 
reading comprehension. 

6. The Project has made some outreach to parents. A formal 
presentation of the Project was made at two P.T.A. meetings, one 
at each Project School. Parents from one Project classroom 
attended a Potlatch held in connection with a study of Indians. 
Evidences of multiplier effects on parents include r 

a. Four telephone calls to the Office of Evaluation, Columbus 
Public Schools from parents requesting information related 
to the Project. 

b. Nine parents from another school area visited one Project 
School stating that they had heard of the Project and were 
considering moving into the area so that their children 
could attend it. 

7. One of the Directors of the Project has written an article about 
the Project, "The Giant Stirs," which was published in The 
Junior League Topics , March, 1971. 

8. Potential multipliers are anticipated through future roles of the 

Teaching Associates (Graduate Students) involved in the project. 

One will help conduct a workshop this summer on Science in the 

informal Classroom. Two who are currently completing their 

doctoral programs, have taken teaching positions in other 

universities to begin this Autumn (one at the University of 

Rhode Island, the other at Penn State University) . 

NOTE: There were no outside teachers or staff brought into the Project. 
Neither ware teachers outside of the Project group used as 
O consultants. 
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2. Project Director's Report for Seawell Elementary School 

Language Arts/Reading Project — Dr. paul Pritchard, Project Director. 

Five educational communities participated in this project. 
A total of 9 professors from the five universities in the vicinity 
which participated — Duke University, University of North Carolina/ 
Chapel Hill, Shaw University, North Carolina Central University and 
Catawba College — provided on-campus orientation for their students, 
then accompanied them for on-site observation, and concluded by 
assisting the Seawell staff with discussion of observations. Addi- 
tional university involvement occurred with the project director 
visiting four university classes to lecture on the model school emd 
its impact in the area of language arts. University consultant help 
was also utilized for inservice training of Seawell faculty in the 
following areas: organizational patterns, diagnosing, behavioral 
objectives, student self -concepts, language development through 
learning centers, and development of reading programs. 

Approximately fifty graduate students were involved 
in the Seav/ell project in the following assignments: 

1) Six Masters of Arts in Teaching candidates were 

placed in the open classrooms at Seawell and assumed 
major teaching responsibilities functioning as vital 
members of teaching teams. Team leaders and the 
principal assumed supervisory responsibility and con- 
ducted an evaluation of their abilities for the 
university. The mats attended weekly faculty meetings, 
receiving valuable inservice direction from these 
^ . progranfts. 
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2) Three graduate externs in Psychology from UNC/Chapel 
Hill received placement in the classrooms at Seawell 
where they identified/ tested , observed and treated 
individual and groups of students with behavior problems . 

3) Three speech therapists .from the Graduate School of 
Education were placed at Seawell where they identified, 
tested, observed and treated in the classroom indivi-* 
dual students with speech difficulties. 

4) The remaining graduate students received orientation, 
observation and follow-'Up on techniques in development 
of communication skills* 

Undergraduate students involved in the project numbered 115 
and assisted in the following capacities: 

1) In cooperation with LINC, six male undergraduates 
from area universities participated as interns in open 
classrooms. These interns gained valuable observation 
experience, helped teachers develop new programs, pro- 
vided individual attention for students and organized 
art projects, nature hikes, athletic activities and 
others. 

2) Again in cooperation with the University of North 
Carolina, two undergraduate practice teachers were 
utilized as practice teachers, one majoring in art 
education and one majoring in physical education. Both 
students established outstanding programs in their 
areas of interest and were able to observe and function 
effectively in a model school. 
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3) The remaining number of undergraduate students 

received orientation, observation and follow-up on 
techniques in operating a language development pro- 
gram in the open classroom. 
An overall total of 640 visitors came to observe at Seawell 
during the 1970-71 school year. They represented a diverse group 
of professionals^ including: administrators (superintendents, 
assistants, etc.) , county grand jury members, elementary school 
teachers (local and state-wide) , college students (graduate and 
undergraduate), teachers aide trainees, parents, community visitors, 
school board members, and one congressional aide. 

Of the overall total of 640 visitors, 379 were outside 
teachers brought into the project throughout the year. These par- 
ticipants received initial inservice training at their own schools, 
focused around the identification of individual student needs, devel- 
opment of communication skills, classroom organization and independent 
learning. While at Seawell their four-day sessions were 
divided into observation/participation in the classrooms (i.e., 
diagnosis, effective reading instruction, teacher-made materials), 
with all activities focusing on language develo^mient activities, 
Following on-site observation these teachers received inservice 
follow-up programs at their individual schools. Time was spent in 
each teacher's classroom providing assistance to the teacher in the 
development and implementation of a total language program. 

In addition 5 teachers were brought in from outside the 
project group. These consultants conducted inservice workshops for 
* Seawell teachers, providing classroom observation and consultation 
ERJC them. 
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Of the total 150 families who had children in the Seawell 
project, all were involved to some degree at one time or another. 
Two formal parent organizations existed / one appointive membership 
(PAC) and the other elective (PTA). The Principal ' s Advisory 
Council was composed of 18 parents appointed by the principal. They 
were selected as representative racially and geographically of the 
student population. The Council served as a major source of commu- 
nication between parents and administration with the Council assuming 
responsibility for passing on their knowledge gained to other members 
of the school community. There were 5 meetings during the 1970-71 
school year. The PTA was organized along functional lines with 
activities following usual PTA duties* There were 7 meetings through- 
out the year with major emphasis on information and socialization. 

On a more individual level, major contact was established 
with each family by a home visit. All Seawell families were either 
visited or given the option of declining a visit. The program was 
explained to the families with notations entered into a master card 
file system as to the families' availability to work in the school. 
In the spring, every Thursday morning was open to parents for obser- 
vati on -orientation in the classroom. An estimated 50 Seawell parents 
took advantage of this. 

Parents were utilized on regular schedules to work in the 
classroom and around Seawell. Four mothers regularly attended 
Friday workshops to develop teacher materials while an estimated 20 
parents were utilized in the classrooms at varying times. Their 
activities included establishing interest centers ixnder teacher 
direction, aiding with individual students, field trips, class par- 
^^Bf and teacher aid duties. Four members of the Seawell faculty 
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were also parents and one assistant teacher was a parent. 
There were approximately 15 Saturday mornings when 3-10 fathers 
(and occasionally mothers) would work on developing the school 
playground., 

3. Project Director's Report for the Temple University PIRLT 
Project^ Philadelphia y Pennsylvania — Dr, Howard Blake ^ Project 
Director. 

Temple University professors and students were directly 
involved in this project. Two senior professors served as the 
project leadership, and 5 professors acted as consultants and seminar 
leaders. Two graduate students performed duties as full-time assis- 
tants serving as demonstration teachers and resource persons and 
developing instructional materials. Additionally, 40 graduate 
students participated as teachers and received ten semester hours 
of graduate credit for involvement in the program. Undergraduate 
students, also totaling 40, assisted as student teachers in class- 
rooms of participating teachers. 

Among the visitors participating in the project were: 
1 Philadelphia Public School Board member; 4 visiting teachers who 
served as consultants in special areas; cind 24 others, whose posi- 
tions include: reading and language arts supervisors of the school 
district, staff members at Research for Better Schools, the reading 
and language arts supervisor of Delaware County schools, teachers 
in Philadelphia schools, and faculty members from Temple « This 
group of visitors did not include directors of the other EPDA 
projects and the teachers from Columbus who came with their director. 
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The project staff presented the dimensions of the project 
t to various education professional audiences, including: three 
meetings of the Citizens Commission on Public Education in Phila- 
delphia^; two meetings of the elementary school principal groups in 
Philadelphia; one group of student teaching supervisors at Temple; 
one National Conference on Lemguage Arts in Los Angeles; arid one 
meeting of the Pennsylvania Research Association. Project staff 
was also scheduled to present the dimensions at the Pennsylvania 
Council of Teachers of English conference in October, and at the 
National Council of Teachers of English pre-convention workshop in 
November. 

Parents of students in the participating schools also became 
involved in the project. Twelve parents were trained as aides to 
assist teachers in the reading/language programs. These parents 
were also trained to organize small seminars of other parents to 
teach them how they can assist children at home with learning to read. 
Approximately 50 parents volunteered to participate in these seminars. 
The 12 parent aides met regularly on Thursday mornings from 9 to 12 
for a seminar or training session. From September through February 
they met weekly; March through June, bi-weekly. A total of 22 
three-hour sessions were held. The parent aides as a group did not 
meet regularly with the group of participating teachers. They plan to 
correct this situation next year. These parent aides also worked 
in the classrooms of an assigned participating teacher one day per 
week on a scheduled basis. They also spent one-half day per week 
in the volunteer seminars for the other parents. 

Evidence of parent participation is seen in the following 
aspects of their partinioation: 
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1) The attendance records of parent aides at the 
training sessions was regular: average attendance 
was nine parents* 

2) Attendance by parent aides in the classrooms was 
consistently regular • 

3) Parent aides were continually utilized in the 
project from the beginning of the year to the end* 
Of the 14 original parent aides, ten continued 
throughout the year, 

4) In general, the attendance of parents at volunteer 
seminars in homes was good* 

The project director was asked to summarize the evidence of 
multiplier effect. The following are the various areas in which the 
project is considered to have been influential in its impact: 

1) The two Temple professors assigned to the project will 
hold joint appointments with the School District for 
next year to continue giving leadership to the project, 

i 

to teach a course for reading and language arts super- 
visors of the School District, to consult workshops/ 
and to serve as consultants on the reading and language 
arts programs — using in all this work the approach 
developed in the project* 

2) One of the doctoral students who served on the staff 
this year has taken an appointment for next year as a 
master teacher at Edison High School in a project that 
has many similarities to the PIRLT project, enabling 
her to extend the PIRLT approach to that project* 
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3) During the summer of 1971, two members of the pro- 
ject staff developed a booklet putting forth the 
main strands in PIRLT, to be disseminated throughout 
the School District and utilized for modeling other 
language/reading programs after PIRLT, 

4) During the summer of 1971, some members of the pro- 
ject staff prepared instructional materials that 
were developed in PIRLT for dissemination throughout 
the School District, 

Although evidence of the achievements of the project are 
tentative at this point, the School District feels satisfied enough 
that it has funded the project for another year, with allocations 
for two language arts/reading supervisors, the continuation of the 
services and training of 14 parent aides, additional instructional 
materials, and the clerical assistance necessary. 
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CHAPTER V 

RESULTS OF TEACHER PRACTICES AND ATTITUDE SURVEY 
Introduction 

Two of the major goals of the five NCTE/USOE projects 
concerned the attitudes of project participants and their teaching 
practices in the language arts/reading areas. If the centers were 
effective, teaching practices should change toward the use of more 
innovative and a greater variety of techniques and practices • Fur- 
ther / if the projects were effective, it seemod reasonable to assume 
that the attitudes of teachers toward the schools/ their administra- 
tions and their students would positively change. 

Appendix C presents a copy of the Teacher Practices and 
Attitude Survey developed by the SPECS evaluation team. It should 
be noted that each of the project directors provided, first, a series 
of ideas about the survey, and, finally, reaction to the first draft. 
The survey was scored in three subscales — teacher attitudes, teach- 
er practices and a miscellaneous field — that allowed us to tap 
ideas and feelings teachers had about their effectiveness in dealing 
with disadvantaged children and the teaching of non-standard dialects. 
Open-ended questions concerning priorities given to certain language 
arts teaching allowed us to tap still further the ideas of partici- 
pating teachers. 

Attitudinal change is an important attribute in assessing 
the success or failure of an innovative program. Research has shown 
that the attitude that one holds toward an innovation or program 
will determine in large part whether or not the innovation is adopted. 
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Researchers have shown that the individual attribute in personality 
characteristics is so important that one study labeled 
individuals on a continuum from Innovators to Laggards: Innovators 
being those that readily accept an innovation and are somewhat posi- 
tive of success, and Laggards being those individuals who involve 
themselves in innovative activities only from a reserved or pessi- 
mistic point of view» One need not belabor the fact that attitudes 
are important to the success or failure of any program, but simply 
state that a measure of attitude and a subsequent change in attitudes 
over a period of time can be a useful tool in helping evaluate a program. 

The Role of Evaluators 

Professional evaluators frequently are called upon to 
evaluate programs with technique3 ranging from personal observations 
to controlled research studies done by using complex, statistical 
tools in a controlled environment with carefully selected groups of 
subjects. In studying education goals and practices, evaluators 
are frequently called upon to examine a group of ideas and come forth 
with an orderly presentation of data that can be used by the decision- 
making unit of a school or agency to make reasonable judgment about 
future planning. This is not always a simple task, especially in 
light of the many needs and desires of the diverse population usually 
examined. The role of the evaluator is, therefore, a unique one, but 
one which is basically concerned with helping teachers and adminis- 
trators define goals for pupils, learn to discover differences among 
pupils and teachers, and design programs to find out whether or not 
instructional procedures are good. 
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Problems of Evaluation 

Upon closer examination of evaluative studies, one realizes 
that most research data is only a single bench mark taken as a 
description of a process, phenomenon or attitude at the time and 
is not equivalent to understanding the total person or dynamics of 
a group. Unless the process is static, something quite rare in 
human behavior, predicting future behavior points at ...x + 1, 
X + 2...X + n... based on an observation point at x is risky. If we 
are not willing to make assumptions about the man and his social 
arrangements, such prediction may be impossible. Considering this 
fact in the evaluation of this program, we sought information at two 
different periods of time. The problem associated in using the same 
instrument twice was, of course, considered. But these objections 
were superceded in order to survey the attitudes and feelings of 
those responding to the instrument, as well as to assess changes in 
these attitudes. 

Another problem in utilizing data effectively for evaluative 
purposes is the problem of presenting the data in a useful manner. 
Unfortunately, there is a frequent lack of relevant- information in 
research data« While researchers, by the nature of their training, 
are fascinated by their data, teachers and other program professionals 
are overv^helmed by the number of problems which need change. Research 
data or findings, in short, seem foreign and remote when compared 
to the real world as seen by administrators, reading specialists 
and teachers. 

If educational research demonstrates that the learning rate 
of public school children, randomly selected and enrolled in a 
^oecial reading program funded for a half-million dollars each year, 
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is not significantly higher statistically than a matched subpopula- 
tion not in the special program, what administratively should result? 
Abolish the program and write off the year's expense or continue 
the program for another year? What if the program is in its first 
of three years of promised funding and the federal representative is 
very interested? Complexity of decision increases if this program 
is directed at underachievers, for even if the program is ended the 
problem still remains. A frequent administrative response to such 
findings, assuming this program is not itself a public issue, is to 
ignore the data and continue. Thus, the survival of the program can 
override the insight of the data. 

Responses to Subjective Items 

The survey questionnaire administered to participants in 
the NCTE/EPUA project allowed teachers to respond to open-end ques- 
tions concerning effective techniques and objectives for reading 
programs and to state how individual competence in teaching reading 
could be improved. Participants listed in rank order the ten 
(usually less) most effective techniques or activities for teaching 
reading. They also listed in rank order ten objectives for their 
reading program. 

The pre-test results of the survey revealed that teachers 
were using a variety of instructional techniques. The two most fre- 
quently occurring ones were use of the language experience approach 
and individualized reading. Phonics was another popular method* 
Other techniques that were mentioned included basal readers, manuals 
and workbooks, S.R.A. materials and diagnostic testing. The language 
O 3erience approach was the most frequently mentioned technique. 
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When asked to list in rank order ten objectives for the 
reading program, many respondents listed fewer than ten objectives. 
Again, there was a wide variety of responses. The most frequently 
occurring objectives were to get the child to enjoy reacting and to 
develop an appreciation for reading. Other objectives ranked as 
number one centered around the building and development of certain 
basic reading skills dependent on the child's level of ability. 

Many respondents to the questionnaire seemed at a loss to 
answer the question of how they could improve their competency in 
the teaching of reading. Some of the ways mentioned dealth with use 
of teacher aides and the availability of materials related to the 
activities and needs of children. Other responses included knowledge 
of many techniques for the teaching of reading, workshops to gain 
new ideas, and better diagnostic means of measuring the child's ac- 
hievement. 

The post- test results of the survey did not show measurable 
change. Most responses listed on the pre-test survey were again 
mentioned the second time. Again, the most frequently occurring 
responses to the question of effective techniques to teach reading 
were the language expe-.ience approach and individualized instruction. 
Other techniques were basal readers, flash cards and phonics acti- 
vities. 

Objectives for the reading program listed on the post-test 
survey centered around increasing the reading ability of each child. 
Again, one of the most frequently mentioned objectives included get- 
ting the child to enjoy reading. 

Although many kinds of activities occurred during the NOTE/ 
^^OA projects, the responses to the three narrative^type questions 
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on the survey showed little change. It is difficult to determine 
whether this lack of change was a result of the way the questions 
were worded or the projects themselves. Other indications of change 
can be gleaned from other questions on the survey. 

Activities or techniques used in the language arts program 
tended to favor the language experience approach. However^ the change 
was not significant. Again ^ a variety of techniques were listed. 
The question intended to identify the means by which teachers could 
increase their competence in the teaching of reading brought fewer 
responses on the post-test than on the pre-test. 

The responses to the three subjective questions on the 
Teacher Practices and Attitude Survey indicate little change. One 
plausable explanation would be that the pre-test questionnaire was 
administered after the teacher had participated in pre-school workshops 
conducted by the project directors. The most frequently mentioned 
practices and objectives on both the pre-test and post-test closely 
paralleled the most frequently mentioned practices and objectives 
of the projects. The positive correlation of pre*-test/post-test 
results on this part of the survey with the objectives of the pro-- 
grams as listed in the proposals would seem to indicate that the 
workshops produced positive results. 

Sample 

The Teacher Practices and Attitude Survey was completed by 
a total of 158 teachers and administrators. Table 1 presenti^ the 
distribution of subjects by center and by position held. Table 2 
presents the same data for those subjects whose surveys were com- 
pleted both pre and post and in suitable fom for analysis. 
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TABLE 1 

THE DISTRIBUTION OF TOTAL SUBJECTS 
RESPONSE BY POSITION AND LOCATION 



Location 



Administrator Teacher Total No. Total % 



Berkeley 
Chapel Hill 
Columbus 
Philadelphia 
Portland 



0.0 
0.0 
6.1 
0.0 
7.0 



96.4 
100.0 

93.9 
100.0 

93.0 



27 
28 
33 
27 
43 



17.1 
17.7 
20.9 
17.1 
27.2 



Total Number 
Total Percent 



3.2 



153 
96.2 



158 



100.0 



TABLE 2 

THE DISTRIBUTION OF DATA USED IN STUDY 
BY POSITION AND LOCATION 



Location 



Administrator Teacher Total No. Total % 



Berkeley 
Chapel Hill 
Columbus 
Philadelphia 
Portland 



0.0 
0.0 
10.0 
0.0 
3.6 



100.0 
100.0 

90.0 
100.0 

96.4 



16 
6 

29 
25 
28 



15.4 
5.3 
27.9 
24.0 
26.9 



Total Number 
Total Per::ent 



3 
2.9 



101 
97.1 



104 



100. Q 
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The questionnaire consisted of three major sections: 1) 
attitudes toward general school practices, 2) frequency of use of 
various teaching techniques, and 3) attitudes toward parent involve^ 
ment and non-standard dialects. 

In coding the responses pre- and post-test results were 
transferred directly from response sheet to punch card, with the 
exception of the open-ended questions in Part 3, Because of the 
open-ended nature of some of the responses in Part 3, they were 
included in the earlier analysis. This left 66 questions which were 
used in analyzing the differences between pre- and post-test 
responses, and among the various centers. In order to determine 
whether or not significant differences existed among the centers on 
either the pre- or post-tests, between the pre- and post-test scores, 
or whether there was an interaction between the way individuals in 
the various centers reacted from pre- to post- test, a Factorial 
Design Analysis of Variance was completed. 

The pre- and post-test means of the three sections of the 
Attitude Survey are presented in Table 3. It may be observed that 
the Chapel Hill, Columbus and Portland sections had the highest 
pre-test attitude scores, and were very homogeneous with respect to 
pre-test attitude scores. The Berkeley and Philadelphia centers 
had somewhat lower pre-test attitude scores. A high score on Section 1 
of the Attitude Survey indicates agreement with the positive attitudes 
stated in the survey. Examination of the post- test scores on Section 1 
of the survey reveals that all of the centers, except Berkeley, had 
lower attitude scores at the time of post-test than they had evidenced 
at the time of pre-test. The Berkeley center, on the other hand, had 
a higher level of agreement with the positive statements of Section 1 
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TABLE 3 

DISTRIBUTION OF CELL MEANS BY CENTER LOCATION 



Location 
Test 1: 

Berkeley 

Chapel Hill 

Columbus 

Philadelphia 

Portlemd 

Test 2: 
Berkeley 
Chapel Hill 
Columbus 
Philadelphia 
Portland 

Test 3: 
Berkeley 
Chapel Hill 
Columbus 
Philadelphia 
Portland 



Cell Means 

Pre-Test Post-Test 

164.06 178.62 

185.75 176.75' 

183.61 169.43 

175.58 169.29 

184.00 180.96 



167.12 
167.62 
167.54 
177.54 
177.54 



164.75 
145.87 
172.04 
178.58 
182.04 



125.06 
131.25 
125.96 
129.62 
126.32 



127.50 
121.12 
124.11 
124.17 
128.54 
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at the time of post-test than had been evident at the time of pre- 
test. Some decline in attitude might be expected in any Septembers- 
May comparison. 

Viewing the pre-test means of Section 2 on the Attitude 
Survey, it may be observed that Philadelphia and Portland showed 
the greatest frequency of use of the teaching method indicated in 
the survey, with Berkeley, Chapel Hill and Columbus centers showing 
somewhat less frequent use of the method included in the survey. 
Examination of the post-test means for Section 2 indicates that 
Columbus, Philadelphia and Portland centers increased in their uti- 
lization of these teaching techniques, while both the Chapel Rill 
and Berkeley centers showed less utilization of these teaching tech- 
niques at the post-test than they had at the pre-test. 

Statistical Analysis 

Table 4 presents a summary of the Factorial Design Analysis 
of Variance of the pre- and post-test scores in the five centers 
for each of the three sections of the survey. It may be observed 
that the differences between the pre- and post-test scores, the 
differences between centers and the interaction are all statistically 
significant for Section 1 of the survey. There is a significant 
decline in agreement with the positive statements u£ Section 1 
overall; there are differences among the centers in the amount of 
agreement with the positive statements of Section 1; and significant 
interaction exists in that the Berkeley center showed greater agree- 
ment at the time of post-test, while all others showed lower agree- 
ment at the time of post-test. 
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TABLE 4 

ANALYSIS OF VARIANCE FOR FACTORIAL DESIGN WITH 
UNEQUAL CELL FREQUENCY, LEAST SQUARE SOLUTION 
BY PRE/POST TEST AND LOCATION OF CENTER 



Variable 



Test 1: Attitudinal Ques- 
tionnaire, General and 
Special Educational 
Reading Concepts 



Location of Centers 
Pre/Post Test 
Interaction 
Error 



P 

6.5482 
7.0223 
7.1593 



df 
4 

1 

4 

198 



Significance 
.01 

.01 

.01 



Test 2: Survey of Frequency of 
Use of Teaching Techniques ^ 

Location of Centers 6.7653 

Pre/Post Test 0.0459 

Interaction 1.4097 

Error 



df 
4 

1 
4 

198 



Significance 
.01 



Test 3: Effectiveness in 
Dealing with Disadvantaged 
Children £ 



Location of Centers 
Pre/Post Test 
Interaction 
Error 



0.6683 
1.9431 
3.0510 



df 

4 
1 
4 

198 



Significance 
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With respect to Section 2 of the survey, only the differences 
between centers were statistically significants The differences 
between pre- and post-test scores are probably chance differences. 
The frequency of utilization of the techniques included in the survey 
was significantly higher at the Philadelphia and Portland centers 
than at either the Berkeley or Chapel Hill centers. 

With respect to Section 3 of the survey, none of the differ- 
ences observed in the pre- and post-test scores or between the centers 
was statistically significant. 

Summary 

There were significant differences cunong the centers in 
mean scores earned by teachers at the various centers on Test 1 i 
(Attitudes) of the Teacher Practices and Attitude Survey • The 
Chapel Hill, Columbus and Portland centers shov;ed greatest level 
of agreement with the survey items, with the Berkeley ^^nd Philadel- 
phia centers showing less agreement, A decline in agreement with 
survey items was found for each center, except Berkeley, which 
showed a significant increase in agreement with survey items. Since 
agreement on these items is considered a positive orientaL:* jn, some 
decline in positive orientation, some decline in positive orientation 
may be inferred, except for Berkeley. This might be seen as fairly 
typical of a September-May change. 
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CHAPTER VI 

RESULTS OF THE STUDY OF PUPIL TEST PERFORMANCE 

The analysis of pupil achievement and attitude performance 
will, in this chapter, be reviewed on a center by center basis. 
Because each center selected its own assessment battery it was not 
possible to study performance of students across centers. Further 
we were unable to include the Portland Center students in this 
section of the study. 

The following tables, figures and discussions, therefore, 
take each center in sequence and review the accomplishments in 
pupil performance. They are reviewed in the following sequence: 

Philadelphia 

Columbus 

Chapel Hill 

Berkley 
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Philadelphia/Temple 

Pre and post-test scores were available for at least some 
grade levels at each of the centers except Portland, Because control 
group.'i were not utilised and different tests were used at the various 
centers f rigorous between-center inference is not possible. In order 
to make it possible to identify the salient performance of students 
and make some inferences about relative strengths, the test data are 
presented both in tabular and graphic formats. 

Table 1 to 6 presents the mean pre- and post-test scores of 
students in grades 1-6 at the Philadelphia center/ Average change 
scores and change scores presented in £ score form are also included 
in Table 1-6. The use of ^ scores makes it possible to compare rela- 
tive performance across scales and to a limited degree between 
centers. Figure 1 presents the data contained in Table 1 in graphic 
format. Only subjects who completed both pre- and post-tests were 
utilized. The pre-test performance is presented as the baseline 
with post-test performance plotted as deviations from the pre-test 
line. The standard deviation of the pre- and post-test groups was 
used as the basis for computing the z_ score deviations. It may be 
observed from Figure 1 that grade 1 students in the Philadelphia 
project earned higher mean scores on the post-test on each of the 
measures. It may also be observed that the greatest improvement 
was on the copying, matching and alphabet scales v/ith somewhat lower 
gain in mean z scores on the word meaning and listening tests. 

Figure 2 presents the same data for the scales on which 
pre- and post-test data were available at grade 2. It may be ob- 
served that while both reading comprehension and vocabulary increased, 
ERJC ^ 2 score change was greatest in vocabulary. 
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Table 3 presents the data for grade 3 and reveals improve- 
ment only on the vocabulary measure, with little change in the other 
measures « 

Table 4 presents fourth grade data from the Philadelphia 
project. Improvement may be noted on all the achievement measures, 
while little change occurred in the attitude measures. 

Table 5 presents the data for grade 5 in the Philadelphia 
project. Again, it may be observed that improvement occurred in 
all achievement scales. At grade 5 a positive change also occurred 
in the attitude scores, with the exception of the attitude prefer- 
ence for reading aloud scale. 

Table 6 presents the scores for grade six, revealing only 
modest improvement in reading comprehension but good improvement in 
other achievement measures. The attitude measures show a decline, 
except for the measure of confidence in reading which shows an 
increase in confidence. 
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TABLE 1 

GRADE 1 - Philadelphia/Temple 
Pre-Mean, Post-Mean, Change and z Scores 



SOBSCALE 


Pre-Mean 


Post-Mean 


Change 


z 


Word Meaning 


5.84 


6.88 


1.04 


.42 


Listening 


8.18 


9.51 


1.33 


.51 


Matching 


4.45 


9.51 


5.06 


1.45 


Alphabet 


7.53 


14.86 


7. .33 


1.92 


Numbers 


8.65 


13.37 


4.72 


1.29 


Copying 


4.49 


9.84 


5.35 


1.52 


Total 


38.82 


63.96 


25.14 


1.70 



1 
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Figure 1 
Grade One Philadelphia/Temple 
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TABLE 2 

GRADE 2 - Philadelphia/Temple 
Pre-Mfian, Post-Mean, Change and z Scores 

SUBSCALE Pre-Mean Post-Mean Change z 

Reading 4.36 ^.69 1.33 .21 

Vocabulary 44.05 57.68 13.63 .70 
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Figure 2 
Giirade 2 - Philadelphia/Temple 
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TABLE 3 

GRADE 3 - Philadelphia/Temple 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE Pre-Mean 
Reading 9.94 


Post-Mean 
10.02 


Change 
.08 


z 

.01 


Vocabulary 64.98 


71.88 


6.90 


.50 


Total Reading Attitude Score 21.62 


21.49 


-.13 


-.03 


Attitude: Liking for 

Reading 9.84 


9 . 61 


-.23 


-.09 


Attitude: Confidence in 

Reading 6.92 


6.80 


-.12 


-.08 


Attitude: Preference for 

Reading Aloud 5.16 


5.07 


-.09 


-.05 
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Figure 3 
Grade 3 - Philadelphia/Temple 
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TABLE 4 

GRADE 4 - Philadelphla/Temi>le 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 


Pre-Mean 


Post-Mean 


Change 


z 


Reading 


19.88 


22.84 


2.96 


.27 


Vocabulary 


14.20 


22.63 


8.43 


.72 


Auditory Discrimination 


22.21 


28.22 


6.01 


.64 


Syllabication 


9.82 


10.97 


1.15 


.28 


Beginning and Ending 
Sounds 


19.41 


22.33 


2.92 


.41 


Total Reading Attitude 
Score 


21.38 


20.39 


-.99 


-.18 


Blending 


13.34 


14.91 


1.57 


.31 


Sound 


15.44 


19.48 


4.04 


.47 


Attitude: Liking for 
Reading 


10.04 


10.18 


.14 


.04 


Attitude: Confidence in 
Reading 


6.46 


6.57 


.11 


.06 


Attitude: Preference for 
Reading Aloud 


4.60 


4.37 


-.23 


-.12 
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Figure 4 
Grade 4 - Philadelphia/Temple 
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TABLE 5 

GRADE 5 - Philadelphia/Temple 
Pre-Mean, Post-Mean, Change and z Scores 



SOBSCALE 


Pre-Mean 


Post-Mean 


Change 


z 


Reading 


19.18 


27.95 


8.77 


.83 


Vocabulary 


13.20 


19.84 


6.64 


1.09 


Auditory Discrimination 


22.63 


29.20 


' 6.57 


.58 


Syllabication 


9.57 


12.14 


2.57 


.61 


Beginning and Ending Sounds 


18.47 


22.79 


4.32 


.50 


Total Reading Attitude Score 


22.29 


23.80 


1.51 


.34 


Blending 


11.20 


15.75 


4.55 


.81 


Sound Discrimination 


15.91 


22.77 


6.86 


.75 


Attitudes Liking for 
Reading 


10.44 


12.11 


1.67 


.56 


Attitude: Confidence in 
Reading 


6.32 


6.67 


.35 


.24 


Attitude: Preference for 
Reading Aloud 


5.37 


5.27 


.10 


.04 
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. Figure 5 



Grade 5 - Philadelphia/Temple 
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TABLE 6 

GRADE 6 - Philadelphia/Temple 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 


Pre-Mean 


Post-Mean 


Change 




Reading 


31.46 


32.00 


.54 


.06 


Vocabulary 


19.25 


27.38 


8. 13 


.71 


Auditory Discrimination 


31.75 


37.40 


5.65 


.66 


Syllabication 


13.29 


14.95 


1.66 


.37 


Beginning and Ending Sounds 


27 . 0/! 


30.80 


3.76 


.62 


Total Reading Attitude Score 


21.68 


19.83 


-1.85 


-.31 


Blending 


16.83 


17.10 


.27 


.05 


Sound Discrimination 


25.63 


30.60 


4.97 


.72 


Attitude: Liking for Reading 


10.36 


8.92 


-1.44 


-.50 


Attitude: Confidence in 
Reading 


5.32 


5.46 


.14 


.65 


Attitude: Preference for 
Reading Aloud 


6.05 


5.29 


-.76 


-.31 
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Figure 6 
Grade 6 - Philadelphia/Temple 
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Ohio State/Columbus 

Tables 7-10 present the pre- and post-test scores of 
students in grades 1-4 at the Columbus^ Ohio State center. Figures 
7-10 present the same data contained in the tables in graphic format. 
The pre-test performance is presented as the baseline with post-test 
performance plotted as deviations from the pre-test line. The 
standard deviation of the pre- and post-test groups was used as 
the basis for computing the z score deviations. It may be observed 
from Figure 1 that grade 1 students in the Ohio State Project showed 
very little change. Their attendance did improve but not signifi- 
cantly. The reading attitude post-scores were below the baseline. 

Figure 8 presents the same data for the scales on which pre- 
and post-test data were available at grade 2. It may be observed 
, that while reading comprehension and vocabulary inproved on the 
post-test, reading attitude showed a decline when compared to the 
pre-test. Attendance shows the greatest improvement among all 
variables . 

Figure 9 presents the data" for grade 3 which reveals that 
reading comprehension has the greatest gain with vocabulary / reading, 
attitude and attendance also showing gains. 

Figure 10 presents the data for grade 4. It may be observed 
that there was improvement on the reading comprehension measure with 
little change on the other measures. 
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TABLE 7 

GRADE 1 *• Coliunbus/Ohio State 
Pre-Mean, Post-Mean, Change and z Scores 

SUDSCALE Pre -Mean Post-Mean Change z 

CSU Reading Attitude Inventory 388.40 385.90 -2.50 -.04 

Attendance 1969-70 148.04 157.82 9.78 .34 

Days Absent 1969-70 20.84 15.53 -5.31 -.34 
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Figure 7 
Grade 1 - Ohio State/Columbus 
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TABLE 8 

Grade 2 - Columbus/Ohio State 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 


Pre-Mean 


Post-Mean 


Change 


z 


Reading Comprehension 


7.28 


9.32 


2.04 


.39 


Vocabulary 


58.91 


69.93 


11.02 


.29 


OSU Reading Attitude Inv. 


391.58 


376.57 


-15.01 


-.21 


Attendance 1969-70 


147.30 


161.18 


13.88 


.48 


Days Absent 1969-70 


12.17 


13.02 


.85 


.07 
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Figure 8 
Grade 2, Ohio Sta to/Columbus 
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TABLE 9 

Grade 3 - Columbus/Ohio State 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 


Pre-Mean 


Post- 


•Mean 


Change 




Reading Comprehension 


15.10 


20. 


02 


4.92 


.52 


Vocabulary 


11.70 


15. 


17 


3.47 


.43 


OSU Reading Attitude 












Inventory 


360.28 


363. 


18 


2.90 


.05 


Attendance 1969-70 


156.65 


164. 


14 


7.49 


.27 


Days Absent 1969-70 


11.75 


10. 


35 


-1.40 


-.13 
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Figure 9 
Grade 3, Ohio State/Columbus 
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TABLE 10 
Grade 4 - Columbus/Ohio State 
Pre-Moan, Post-Mean, Change and z Scores 



SUBSCALE Pre— Mean Post-Mean Change z 

14.28 15.74 1.46 .21 

Vocabulary 14.26 13.56 -.70 -.10 

OSU Reading Attitude 

Inventory 340.21 338.72 -1.49 -.02 

Attendance 1969-70 166.57 165.15 -1.42 -.07 

Days Absent 1969-70 10.57 10.82 .25 .02 
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Figure 10 
Grade 4, Ohio State /Columbus 
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Chapel Hill/Seawell 

Tables 11--14 present the mean pre- and post-test scores of 
students in grades 3-6 at the Seawall center. Average change 
scores and change scores presented in z score form are also in- 
cluded in these tables. Figure 11 presents the data contained in 
Table 11 in graphic format. The pre-test performance is presented 
as the baseline with the post-test performance plotted as deviations 
from the pre-test line. The standard deviation of the pre- and post- 
test groups was used as the basis for computing the z score deviations. 
It may be observed from Figure 11 that grade 3 students in the 
Seawell project earned higher mean scores on the post-test on each 
of the measures. It may also be observed that the greatest improve- 
ment was on the arithmetic computation with somewhat lower gain in 
z scores on the language arts related subscales. 

Table 12 presents the same data fro the scales on which 
pre- and post-test data were available at grade 4. It may be ob- 
served that the students in the grade 4 Seawell project earned higher 
mecm scores on the post-test on each of the measures. The greatest 
improvement was on the arithmetic computation with a somewhat lower 
gain in mean z score on the arithmetic concepts test. 

Table 13 prese^its the mean scores on the pre- and post-test 
data of the students in grade 5 at the Seawell center. Higher gain 
scores may be observed in the word meaning, paragraph meaning, spel- 
ling, language, arithmetic concepts, arithmetic application, 
social studies and science tests with the greatest improvement in 
both language and arithmetic concepts and a lower 
mean gain score on the arithmetic ocn^xitation t^st. 

Table 14 presents the scores for grade 6, which reveals 
Jf^' modest improvement in word meaning, paragraph meahing/ spelling, 
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and arithmetic computation tests. Higher mean gain scores may be 
observed on the arithmetic concepts, arithmetic application, socia 
studies and science tests, with the greatest improvement on the 
arithmetic concepts test. 
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TABLE 11 
GRADE 3 - Seawell/LINC 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 


Pre-Mean 


Post-Mean 


Change 


2 


Word Meaning 


30.90 


41.05 


10.15 


.78 


Paragraph Meaning 


29.35 


39.26 


9.91 


.68 


Science and Social Studies 


31.31 


37.56 


6.25 


.50 


Spelling 


29.32 


36.10 


6.78 


.52 


Word Study Skills 


29.74 


33.02 


3.28 


.18 


Language 


31.88 


40.54 


8.66 


.54 


Arithmetic Computation 


24.87 


37.12 


12.25 


1.22 


Arithmetic Concepts 


31.24 


38.80 


7.56 


.54 
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Figure 11 
Grade 3 - Seawell/LINC 
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TABLE 12 
GRADE 4 - Seawell/LINC 



Pre-Mean, Post- 


-Mean, Change 


and z Scores 








SUBSCALE 


Pre-Mean 


Post- 


Mean 


Change 


z 


Word Meaning 


38.49 


46. 


77 


8. 


28 


.63 


Paragraph Meaning 


39.65 


48. 


23 


8. 


58 


.57 


Spelling 


39.19 


46. 


10 


6. 


91 


.59 


Arithmetic Computation 


35.79 


44. 


16 


8. 


37 


.77 


Arithmetic Concepts 


41.95 


45. 


24 


3. 


29 


.22 


Arithmetic Application 


42.73 


49. 


34 


6. 


61 


.44 
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Figure 12 
Grade 4 - Seawell/LINC 
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TABLE 13 
GRADE 5 - Seawell/LINC 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 


Pre^-Mean 


Post— M^»;in 




z 






59 . 50 


8.77 


.46 


Paragraph Meaning 


46.30 


56.66 


10.36 


.54 


Spelling 


49.61 


56.29 


6.68 


.39 


Language 


45.69 


57.04 


18.03 


.88 


Arithmetic Computation 


47.84 


49.16 


1.32 


.08 


Arithmetic Concepts 


42.00 


54.25 


12.25 


1.07 


Arithmetic Application 


48.19 


53.33 


5.14 


.29 


Social Studies 


49.11 


54.00 


4.89 


.31 


Science 


46.30 


56.73 


10.43 


.59 
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Figure 13 
Grade 5 - Seawell/LINC 
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TABLE 14 
GRADE 6 - Seawell/LINC 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 


P re-Mean 


Post- 


Mean 


Change 


£ 


Word Meaning 


68. 


78 


71. 


91 


3.13 


.15 


Paragraph Meaning 


71. 


34 


74. 


25 


2.91 


.11 


Spelling 


62. 


59 


66. 


68 


4.09 


.21 


Arithmetic Computation 


61. 


59 


65. 


31 


3.72 


.17 


Arithmetic Concepts 


50. 


94 


65. 


63 


14.69 


.98 


Arithmetic Applications 


57. 


31 


64. 


43 


7.12 


.37 


Social Studies 


56. 


03 


68. 


97 


12.94 


.56 


Science 


67. 


12 


71. 


91 


4.79 


.19 
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Figure 14 
Grade 6 - Seawell/LINC 
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Berkeley/University of California 

Tables 15-26 present the mean pre- and post-test scores of 
students in grade K-3 at the Berkeley center* Figures 15-18 corres- 
pond to the data from the special sample students and Figures 19-22 
correspond to the students not in the special sample. 

Figure 15 presents the scores from the Reading Readiness 
scale and the Average Number of Words scale for the kindergarten 
students in the special sample. It can be observed that there was 
gain on both measures with the greatest gain shown on reading 
readiness. 

Figure 16 presents the first grade scores on the Reading 
Readiness and Average Number of Words scale on the same group. 
Improvement may be noted on the Reading Readiness scale, but not on 
the Average Number of Wordf^ scale. 

Figure 17 shows the scores for the grade 2 students in the 
Special sample. It may be observed that there was improvement on 
all of the scales with greatest improvement on the ETS Primary Word 
Analysis scale and with the ETS Primary Reading scale and the Written 
Language Rating on Single Picture Sequence scale showing higher im- 
provement than the other scales. The lowest improvement shown is 
on the Average Number of Words scale. 

Figure 18 present:^ the data for the grade 3 students in the 
special sample. Again, it may be observed that improvement occurred 
on all the scales. The greatest improvement was on the Multiple 
Picture Sequence scale. 

The following figures tl9-22) represent the scores of the 
students not in the special sample. Figure 19 shows the gain for 
O he kindergarten students not included in the special sample. It 

ERIC 
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polltan Reading Readlnesa scale. 

Figure 20 shows that the grade 1 students not in the 
special sample also had good improvement on the same scale* 

Figure 21 presents the second grade data from this same 
group of students. Improvement may be noted on all the scales , 
with greatest improvement on the ETS Primary Reading scale and 
the least improvement on the ETS Primary Listening scale « 

Figure 22 shows the data from the third grade students not 
involved in the special sample* It can be observed that there was 
good improvement on all three of the scales. 

The last four figures combine all of the Berkeley project 
data to give an overall picture of all of the students involved; 
both those in the special sample and those not included in the 
special sample. 

Figure 23 presents the kindergarten data. It can be noted 
that there was improverxent on both scales with the greatest gain 
on the Metropolitan Reading Readiness scale. 

Figure 24 shows that there was no gain for the Berkeley 
project first graders on the Average Number of Words scale, but 
there was good gain on the Metropolitan Reading Readiness scale. 

Figure 25 presents the data for all of the Berkeley project 
grade 2 students, both special sample and non-special sample students. 
The data shows that the greatest gain for all of the second grade 
students was on the WLRS and the ETS Primary Reading scales with 
AVWDS scale showing almost no gain. There was some gain, however, 
on all the scales. 

Figure 26 shows that there was improvement on all the 
^"*iles for the Berkeley project third graders as a whole. The 

ERIC 
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greatest improvement was on the Written Language Rating on 
Multiple Picture Sequence scale, and the least gain improvement 
was on the average number of words scale. 
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TABLE 15 
KINDERGARTEN - Berkeley/UC * 
Pre-Mean, Post-Mean, Change and z Scores 

SUBSCALES Pre-Mean Post-Mean Change z 

Metropolitan Reading Readiness 40.78 59.92 19.14 l.oi 

Average Number of Words/ 

Communication Unit 58,20 59.93 1.73 .1131 



Special Sample 
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Figure 15 
Kindergarten - Berkeley/UC 
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TABLE 16 
GRADE 1 - Berkeiey/UG * 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 



Pre-Mean Post-Mean Change z 



Metropolitan Reading Readiness '2.62 85.55 12.93 1.15 

Average Number of Words/ 

Conununication Unit 65.17 64.30 -0.87 -.07 



Special Sample 
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Figure 16 
Grade 1 - Berkeley/UC 
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TABLE 17 

GRADE 2 - Berkeley/UC * 
Pre-Mean, Post-Mean, Change and z Scores 

SUBSCALE Pre-Mean Post-Mean Change z 

Primary Reading 20.17 33.80 13.63 1.02 

Primary Word Analysis 29.23 44.04 14.81 1.24 

Average Number Words/ 

Communication Unit 65.33 66.13 0.80 .07 

Primary Listening 37.37 40.84 3.47 .58 

Written Language Rating on 

Single Picture Sequence 156.07 230.00 73.93 1.07 

Written Language Rating on 

Multiple Picture Sequence 165.19 231.92 66.73 .81 



* Special Sample 
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Figure 17 
Grade 2 - Berkeley/UC 
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TABLE 18 
GRADE 3 - Berkeley/UC * 
Pre-Mean, Post-Mean, Change and z ScoiciS 



SUBSCALE Pre-Mean Post-Mean Change z 

Primary Reading 30.22 33.89 3«67 .30 

Primary Word Analysis 45.84 50.62 4.78 .47 

Average Number Words/ 

Coituttunication Unit 70.37 76.43 6.06 .49 

Primary Listening 32.27 37.88 5.61 .82 

Written Language Rating on 

Single Picture Sequence 236.07 323.33 87.26 .87 

Written Language Rating 

on Multiple Picture Sequence 248.15 336.67 88.52 .94 



* Special Sample 
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Figure 18 
Grade 3 - Berkeley/UC 
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TABLE 19 
KINDERGARTEN - Berkeley/UC * 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCAIjE 



Pre-Mean Post-Mean Change z 
Metropolitan Reading 

Readiness 48.36 63.11 14.75 .80 



Non Special Sample 
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Figure 19 
Kindergarten - Berkeley/UC 
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TABLE 20 
GRADE 1 - Berkeley/UC * 
Pre-Mean, Post- Mean, Change and z Scores 

SUB SCALE Pre -Mean Post-Mean Change z 

Metropolitan Reading Readiness 65.12 78.75 13.63 .96 



* Non Special Sample 
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Figure 20 
Grade 1 - Berkeley/UC 
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TABLE 21 
GRADE 2 - Berkeley/UC * 
Pre-Mean, Post-Mean, Change and z Scores 



SUBSCALE 



Pre-Mean P ost-Mean Change z 

Primary Reading 23.07 36.14 13.07 1.07 

Primary Word Analysis 33.78 42.78 9.00 .74 

Primary Listening 36.45 38.41 1.96 .27 



* Non Special Sample 
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Figure 21 
Grade 2 - Berkeley/UC 
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TABLE 22 
GRADE 3 - Berkeley/UC * 
Pre-Mean, Post-Mean, Change and a Scores 



SUBSCALE 


Pre-Mean 


Post-Mean 


Change 


z 


Primary Reading 


24.99 


35.02 ' 




.95 


Primary Word Analysis 


41.12 


49.78 


8.66 


.89 


Primary Listening 


31.57 


36.13 


4.56 


.61 



* Non Special Sample 
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TABLE 23 
KINDERGARTEN - Berkeley/uc * 
Pre-Mean, Post-Mean, Change and z Scores 

SUBSCALE Pre-Mean Post-Mean changes z 

Metropolitan Reading 

Readiness 46.29 62.21 15.92 .85 

Average Number of Words/ 

Communication Unit 58.20 59.93 1.73 .11 



Total Sample 
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Figure 23 
Kindergarten - Berkeley/UG 
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TABLE 24 
GRADE 1 - BerkAley/UC * 
Pre-Mean, Post-Mean, Change and z Score 

SUBSCALE Pre-Mean Post-Mean Change z 

Metropolitan Pvsading 

Readinesf' 67.03 80.62 13.59 .98 

Average Number of Words/ 

Communication Unit 65.17 64.50 -.87 -.07 



* Total Sample 
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Figure 24 
Grade 1 - *^erkeley/UC 
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TABLE 25 
Grade 2 - Berkeley/UC * 
Pre-Mean^ Post-Mean/ Change and z Sooro 

SUBSCALE Pre-M ean Post-Mean Change z 

Primary Reading 22.23 35.54 13.31 1.06 

Primary Word Analysis 32.46 43.09 10.63 .87 

Average Number of Words/ 

Communication Unit 65.33 66.13 0.80 .07 

Primary Listening 36.70 39.00 2.30 .33 

Written Language Rating on 

Single Picture Sequence 156.07 230.00 73.93 1.07 

Written Language Rating on 

Multiple Picture Sequence 165.16 231.92 66.76 .81 



* Total Sample 
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Figure 25 
Grade 2 - Berkeley/UC 
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TABLE 26 
GRADE 3 - Berkeley/UC * 
Pre-Meanf Post-Mean, Change and z Scores 

SUBSCALES Pre-Mean Post-Mean Chancte z 

Primary Reading 26.17 34.74 8.57 .11 

Primary Word Analysis 42.20 49.97 7.77 ,79 

Average Number of Words/ 

Communication Unit 70.37 76.43 6.06 .49 

Primary Listening 31.72 36.52 4,80 .65 

Written Language Rating on 

Single Picture Sequence 236.07 323.33 87.26 .87 

Written Language Rating on 

Multiple Picture Sequence 248.15 336.67 88.52 .94 



Total Sample 
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Figure 26 
Grade 3 - Berkeley/UC 
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CHAPTER VII 
SUMMARY, ACCOMPLISHMENTS AND CONCLUSION 



CHAPTER VII 
SUMMARY, ACCOMPLISHMENTS AND CONCLUSION 

Specialized Educational Consultant Services, Inc., acted 
as contracted evaluation agency for the National Council of 
Teachers of English to coordinate a national evaluation of five 
language arts/reading model training centers # These centers 
were financed by the United States Office of Education division of 
Professional Educational Development Act through NCTE. Centeics 
were operated cooperatively between public schools and nearby 
universities except for the Chapel Hill center which operated jointly 
with the Learning Institute of North Carolina* Other centers were: 

Berkeley - University of California 

Portland - Portland State University 

ColuiTibus - Ohio State University 

Philadelphia - Temple University 

Each of the five centers had goals and objectives which 
differed from the goals and objectives of the other centers. 
For this reason the uje of a standard pre-post examination schedule 
across projects was not considered feasible. 

The evaluation strategy called for an analysis of within 
center pre-post pupil achievement (this was complete in all but 
one center) and analysis of behavioral change in teacher practices 
and attitudes toward the communication skills areas and toward the 
projects, a report from each project director (three directors 
responded to our request) and site visitation by members of the 
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evaluation team to each center. These methodologies must be 
considered as providing descriptive data from which evaluative 
inferences made by persons concerned with the projects* 

Each center had its special characteristics which reflected 
the concerns and objectives which characterized the individual 
projects • The Ohio State University-Columbus program seemed to 
us to provide the most innovative techniques for teaching and 
learning in the communication skills. Berkeley-University of 
California developed a research personality for the area of 
measurement^ evaluation and research. Among the five centers Berkeley 
certainly worked harder and accomplished more in this total 
research area of language arts/reading. The Seawell Project in 
Chapel Hillr North Carolina, seemed to have the most effective 
model for inservice education and outreach to teachers on-the-job 
in nearby schools. The on-campus program and the follow-up 
supervision for ceachers in the project seemed to stand out in 
Portland along with the emphasis given parents as paraprofessionals 
in the program. Teachers that attended classes on the Portland 
State University campus were given a multitude of ideas for teaching 
reading and language arts. Philadelphia undertook a most difficult 
task: upgrading the language arts/reading opportunities and skills 
of ghetto children. Philadelphia had an outstanding program of 
parent involvement, certainly this would be rated gtmong its 
strengths, along with the efforts made to use an experimental 
program of communication and reading skills for disadvantaged 
children. 
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Reports from the project directors indicated their sense 
of reality in dealing with projects, of this type. They seemed to 
be able to put their projects into proper focus with relation ro the 
school community and the university community. Project directors 
appeared to be able to find a balance between good and innovative 
programs for children and effective training programs for teachers 
without either operating to the detrement of the other • They, 
further, had the foresight and the leadership skills necessary to 
balance national project goals, with goals for their individual 
project. 

. Although the project directors included all of the national 
goals in their programs, in practice they tended to place special 
focus on one goal. Thi© special emphasis may not have been an overt 
action; however # the directors • reports support these findings. 

Results of the Teacher Practices and Attitude Survey 
indicated that Chapel Hill, Columbus and Portland had relatively 
high and homogeneous scores when compared to Philadelphia eind Portland 
at the time of the pre-administration of the survey. All centers, 
except Berkeley, declined in their positive attitude by the spring 
of 1971. The attitude of the Berkeley participants at the close of 
the program were more positive, that is they strongly agreed with 
more items. The Philadelphia and Portland centers showed the 
greatest use of the variety language arts/reading practices surveyed 
at both the beginning and the end of the program. Both Berkeley and 
Chapel Hill showed less frequent use of the practices at the end 
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of the project than at the beginning. The Columbus participants 
showed relatively no change In the frequency with which they 
implemented the practices surveyed, * 

The decline in positive attitudes is not surprising, 
teachers will have a more positive feeling toward their environment 
after returning from a summer vacation than after dealing with the 
problems of education and learning for nine months. 

The analysis of pupil performance data was the most complex 
problem. Philadelphia showed significant gains in all achievement 
areas with the reading comprehension showing the least gain among 
the achievement variables. Student's attitudes, however, showed 
little or no positive change. Columbus pupil performance followed 
a similar pattern with clear gains in each subscale with reading 
comprehension and vocabulary showing the most significant gains. 
Again, there was no positive change in the attitude of the pupils 
as measured by their instrument. Chapel Hill did not use an attitude 
measure; in achievement, however, except for grade six there were 
significant positive gains in all language arts related subscales. 
Berkeley students, overall, showed significant positive gains in 
language related areas especially the primary reading, word 
analysis and listening. The Berkeley special sample, who were 
involved in more extensive measures showed good gains overall 
except in the "Words Per Communication Unit." 

Given the constraint of no comparison control group, 
and the inability to draw many assumptions across a variety of test 
data/ we feel justified in concluding that students enrolled 
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in the four of the five centers (Portland was not involved) 
showed significant positive gains on achievement tests, especially 
in the language arts and reading subscales which were administered. 
Students', as well as teachers', attitudes did not show such gains. 
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APPENDIX A 



ORIGINAL EVALUATION DESIGN FOR THE 
COORDINATED EVALUATION 



SPECIALIZED EDUCATIONAL CONSULTANT SERVICES 



Box 6145 College Station, Durham, N.C 27708 



A Proposal 
for 

The Coordinated Evaluation 
of the 

Five Language/Reading Centers 



A Joint Project 
of the 

National Council of Teachers of English 

and 

The United States Office of Education 



by 

HUv/h I. Peck 
William G. Katzenmeyer 



A PROPOSAL FOR THE EVALUATION OF THE FIVE LANGUAGE/READING CENTERS 
SPONSORED BY THE NATIONAL COUNCIL OF TEACHERS OF ENGLISH AND 
THE U.S. OFFICE OF EDUCATION (EDUCATIONAL PROFESSIONS DEVELOPMENT ACT) 

The National Council of Teacheis of English and the USOE are 
coordinating efforts to establish five language/reading centers throughout the 
United States, Centers are being established in Berkeley, California; Portland, 
Oregon, Columbus, Ohio; Chap^ Hill, North Carolina; and ^Hilaclelphls, Penn- 
sylvania. Although each center is responsible for evaluating the effectiveness 
of its efforts, it is felt that a coordinated evaluation of all five projects v^ould 
be of additional value and would support requests to continue the centers. 

The coordinated evaluation effort poses the following basic goals 
concerning the effectiveness of the reading centers: 

1* To determine if the efforts of the centers increase the 
performance of students — e.g.^ as measured by the 
standardized achievement tests, 

2. To determine which individual activities In the various 
centers hold the greatest promise for broad application, 

3. To determine which of the approaches utilized at the 
various centers toward the realization of a particular 
objective has the greatest yield per dollar invested* 

4. To determine if the efforts of the centers are effective 
in systematically reaching more teachers. (Are the 
centers getting the greatest exposure for the dollar? ) 

5. To determine if the centers are able to change teacher 
behavior in a desirable direction. 

O 
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6. To determine if the centers foster Inter-institutlonal 
cooperation both within the sponsoring agency and 
between agencies that are cooperating* 

7. To determine If the centers are able to provide; new 
materials and methods or new uses for the existing ones* 

The following chart presents some summary data concerning the . 

centers ♦ 





Berk. 


Col. 


Phila, 


Port, 


Chap.Hl. 


Number of Schools 


1 


2 


2 


4 


1 


Grade Levels 


K-3 


Pre K-3 


K-6 


1-8 


1-6 


<• 

Number of Students 


600 


600 


900 


200 


350 


Number of Teachers 


25 


25 


57 


32 


(Team 


Number of Administrators 


2 


2 


2 


4 


In- 


Number of Curr. Spec. 


4 


3 


4 


8 


put) 


Number of Paraprofessionals 


A 




12 


14 


3 


Number of Professional Staff 
(full-time equivalent) 


10 


5 


7 

Parents 
60 


4 


12 



While relative emphasis varies between centers^ five types of 
activities are included. The five activities are: (1) teacher and staff training 
(both pre-service and in-service), (2) community and parent involvementi 
(3) development of Instrumentation, (4) develop and use of materials, (5) the 
development and application of methodology » Each center is working to varying 
degrees with five identifiable populations* students, teachers, parents, 
administrators, and curriculum specialists. 

Th« following model may help to visualize the scheme of the NCTE 



ACTIVITIES 



Training 

Materia ls 
Methods 



Instrum ents 
Community Improvement 



POPULATION 




CENTERS 



One of the most sensitive areas which must be coordinated is that 
of evaluation instrumentation. The evaluation model calls for a pre and post 
observations (or input — output measures) design for each activity with all subject 
populations* Project evaluators should plan for this* The more data collected 
uniformly across all centers, the more effective the coordinated evaluation will 
be. However, it is not necessary that the same Instruments be used at all centers* 
Those who have individual plans for certain tests and othe; instruments should use . 
them. If local districts have a basic testing program for pupils, the coordinated 
evaluation will draw from that program. Data collected for local evaluation efforts 
will, for the most part, be used for the coordinated efforts. Plans should be made 
to forward this data to the coordinated evaluation team* 
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One possibility in the coordinated evaluation design is to relate 
output productivity to program costs. In order to relate output variables to cost. 
It may be desirable to report project budgets on a program basis* This will make 
it possible to evaluate output variables in terms of a cost productivity continuim • 

To complete the coordinated evaluation, the evaluation team will 
need a complete description of the program and a statement of program objectives. 
Since each program would be categorized by activities, we will also need a 
description of activities and a list of activity objectives • Plans could be made 
to convert the present line it^ih l?udget into a program budget. Such a plan will 
permit the study of cost-productivity factors as outlined. 

The NCTE/USOE Language/Reading Evaluation Design which follows 
presents a schematic diagram of the total evaluation design* Input variables 
are characteristics and behaviors of the people and institutions Involved in the 
project. Those characteristics and behaviors that are subject to change as a 
result of project activities are identified as change varlabies (I). Measurement 
criteria are set for each change variable (III). This could be done by each 
center or cooperatively among centers. However, pre-test < observation) data 
must be collected for each change variable. Pre-condition variables are constant 
characteristics related to the program but not likely to change as a result of 
program activities (11). 

Process variables are those variables which measure the extent to 
which the various project activities are functioning efficiently, oconomically, 
and according to plan. The particular process variables involved with each 
activity of the various projects will need to be identified and delineated • Major 
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responsibility for monitoring the process variables identified will lie at the 
project level. 

Process measures will involve both the monitoring of the process 
variables identified, and relating such variables to activity costs. 

Output measures relate directly to those input variables identified 
as subject to change by the project and the criteria set for these measurements. 
These are the post test (observation) data. Preconditions are studies as they 
relate and correlate to success in the program. Output cost accountability 
would include the determination! if possible, of the cost of producing identified 
output. 

The irriplerneiitation of the preceeding design seems to provide the 
most effective and efficient means of evaluating the effectiveness of the five 
language/reading centers being sponsored by the NCTE/USOE Joint venture. 
Its completion will provide rationale for the continued support of the centers 
whose efforts prove successful. 
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APPENDIX B 



DESIGN FORMATS FOR DATA PROCESS FORMS 
AND RELATED CORRESPONDENCE 



NATIONAL COUNCIL OF TEACHERS OF ENGLISH/LANGUAGE READING CENTER 



T^iacher* Data Card Format 



Column 1 



Identification of Center 

1 - Berkeley 

2 - Chapel Hill 

3 - Columbus 

4 - Philadelphia 

5 - Portland 



Columns 2-5 



Column 6 



Teacher ID Number — the last four digits of the 
teachers social security number. 



Sex 



Male 1 
Female - 2 



Column 7 



Race 
Black 
White 
Other 



1 
2 
3 



Columns 8-9 Major Teaching Assignment 



1 
2 
3 
4 
5 
6 



1 
2 
3 
4 
5 
6 



Member of a Primary (K-3) Team 7 

Member of an Intermediate Tecun (4-6) 8 

Other (If used please explain by letter) 
K - 10 



'' Teacher " is used to identify a participant or staff in the 
project* Those receive funding and training from project funds* 



Column 10 



Column 11-26 



Columns 27-40 



Columns 41-49 



RIC 



Pre-Post Code 
Pre test 1 
Post test 2 

(on poRt assessment card only center number and ID 
number and those measures usi3d in post assessment 
need be shown • Use columns as designated. Skip 
others) 

Teacher Attitude Scale 

8 subtests/ two columns each 

Interaction Analysis^-Planders, Taba, Gallagher, etc, 
5 subscores, 2 columns each (please indicate by 
letter subscales used and how punched) 

Critical Thinking Skills (Ashford-Guilford & others) 
(Question & Answer Scales) 
4 subscores/ 2 digits each 

Recall 41-42 

Analytical 43-44 

Integrative 45-46 

Evaluative 47-48 



Columns 50-52 Inventory on Children's Literature 

Columns 53-55 Test of Knowledge-Literary Structure 

Colunms 56-57 Semantic Differential 

Self-Esteem 
Personal Worth 



ERIC 



NATIONAL COUNCIL OP TEACHERS OP ENGLISH/LANGUAGE READING CENTERS 

STUDENT DATA CARD FORMAT 



(Some assessments may be shown that are not used at your center. 
Do not punch those columns) 

Column 1 Identification of Center 
1 Berkeley 

2 - Chapel Hill 

3 - Columbus 

4 - Philadelphia 

5 - Portland 



Columns 2-6 Student ID number 

by each center to 
student should be 
Berkeley: 
Chapel Hill: 
Columbus: 
Philadelphia: 
Portland: 



(a unique 5-digit number assigned 
each student — all data on the 
recorded in this code reference) 
00001 to 02000 
02001 to 04000 
04001 to 06000 
06001 to 09000 
09001 to 11000 



Column 7 Sex 

Male - 1 
Female - 2 



Column 8 



Race 
Black 
White 
Other 



1 

2 
3 



Columns 9-10 



Columns 11-14 



Columns 15-20 



Class Assignment Code 

1- 1 Member of a Primary (K-3) Team 7 

2- 2 Member of an Intermediate (4-6) 

3- 3 Team 8 

4- 4 Other (If used please explain 

5- 5 by letter) 9 

6- 6 K - 10 

Teachers ID number (see columns 2-5 on Teacher Card 
Use Team Leader ID # for student assigned to teams 

Student's Birthday 

2/7/59 is coded 020759; 12/24/58 is coded 
122458 



Column 21 



Pre-test Post-test code 
Pre-test 1 
Post-test 2 

(Note: on post-test card only ID number and those 
measures usad on post assessment need be recorded. 
However^ use coltimns as shown.) 
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Columns 22-25 
Column 26 



Column 27 

Columns 28-30 

Columns 31- 33 

Coixunns 34-36 

Columns 37-39 
Columns 40-42 

Columns 43-45 
Columns 46-48 
Columns 49-51 
Columns 52-54 
Columns 55-57 
Columns 58-60 
Columns 61-70 



Student Mental ability score: 
eg* IQ - 94 coded 0094 
IQ - 107 coded 0107 
Mental age 7-11 coded 0711 
Mental age 10-2 coded 1002 

Code for Scoring Report Form Used for Tests 
*1 - Raw Score (our preference) 

2 - Grade Equivalent 

3 - Percentile 

4 - Stanine 

5 - Other (explain by letter) 



Test Used 

California Achievement Test - 1 

ETS Coop. Test - 2 

Stanford Achievement - 3 

Metropolitan Achievement - 4 

Iowa Test of Basic Skills - 5 

Other (Explain by letter) - 6 



Reading Comprehension 
(5*1 coded 051) 
(11.8 coded 118) 

Vocabulary or similar 



Score or similar subscale 



subscale 



Word Attack Skills (coding) or similar subscale 



Spelling 

Language (English) or similar subscale 



Reading Attitude Score 

Reading Interest Score 

Listening 

Oral or Speaking 

Writing (Carlsen Scale) 

Creativity (Torrance) 

Semantic Differential 
Self-Esteem 
Personal Worth 




SPECIALIZED EDUCATIONAL CO NSULTANT SERVICES 
Box 6145 College Station, Durham, N.C 27708 

December 7, L970 



TO: NCTE Project Director 

FROM: Coordinated Evaluation Team 




RE: Coordinated Evaluation: Progress Report 

In order to keep in closer touch with the five Centers, the 
members of the coordinated evaluation team are apportioning 
responsibility. Each team member will become a liasion j^^erson 
with one of the centers for the national coordinated evaluation. 



Hugh 1. Peck 

Director, Research and Evaluation 
LINC 

1006 Lamond Avenue 
Durham, N. C. 27701 laision 
office (919) 688--4307 for 
home (919) 383-1602 



Mrs. Joan Troy 
Research Assistant 
LINC 

1006 Lamond Avenue 
Durham, N. C. 27701 
office (919) 688-4307 
home (919) 489-7289 

Dr. Robert A^Pittillo 
Professor of Education 
Duke University 
Durham, N. C. 
office (919) 684-3924 
home (919) 489-1012 

Dr. Richard Ray 
Executive Director 
LINC 

1006 Lamond Avenue 
Durham, N. C. 27701 
office (919) 688-8211 
home (919) 929-4216 



laision 
for 



laision 
for 



laision 
for 



Charlotte Houck 
Professor of Education 
Ohio State University 
1945 W, High Street 
Columbus, Ohio 
office (614) 293-2050 
home (614) 267-2786 

Dr. Paul Pritchard 
Seawell Elementary School 
Chapel Hill, N. C. 
office (919) 967-4343 
home (919) 929-6679 



Dr. Robert B. Ruddell 
Professor of Education 
UCLA at Berkeley 
Berkeley, California 



Dr. Colin Dunkeld 
Associate Director 
School of Education 
Portland State University 
P. 0. Box 751 
Portland, Ore. 
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Dr. W. G* Katzenmeyer 
Professor of Education 
Duke University 
Durham, N. c. 
office (919) 684-3924 
home (919) 383-5080 



laision 
for 



Dr. John Pepper 
Project Evaluator 
PIRI.T 

School District of Philadelphia 
Philadelphia, Pa# 



Each of us will try to become more f umilar with your project and 
to keep in close touch with you, Also, we v^ill be in touch 
with you regarding plans for a visit to your project. 

We are now at the stage that pretest data should be arriving 
to us. So far it has not, except for Chapel Hi 11^ 

We need to have the Teacher Practice and Attitude Survey returned 
to us at once . Further, the pre test data process cards for 
teachers and students should be in our hands before Christmas 
Holiday. We have sent to you the Data Card Format for Teachers 
and Students. If you need additional copies contact your laision. 

John Peper has suggested that the following be added to the 
cards. Therefore, if you wish to use these variables please 
use the columns indicated for them. We urge that you add these 
to your data format sheets as tollows: 

Teacher Data Card Format 

Column 58 - Teacher Education Level 

1 = BA 

2 M. Ed. 



1 3 Ph.D. 
Column 59-61 s= Number of Days Teacher was in training 



as part of Project, 
use: 087 



If less than 100 



Column 62 ^ Socio-Economic Status of Teacher 



Student Data Card Format 

Column 71 Scoio-Economic Status of Student 
Column 72-74 4= Number of days attended school 1970-71 

school year. If less than 100 use 087 » 

Please mail all data to us at the letterhead address as soon as 
possible. We are sure you realize the importance of a well 
coordinated evaluation and urge your support. 

HIP:bb 

cc: Robert T. Hogan 
Doris Gunderson 
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APPENDIX C 
TEACHER PRACTICES AND ATTITUDE SURVEY 



NOTE COORDINATED EVALUATION 
Prepared by 
SPECS, INC* 
Durham/ N, C. 
Copyright 1970 

Teacher Practices and Attitude Survey 



Part I - Direations: 

i# Give the last four digits of your 80c?lai security number , 
(Through this technique responnes' cannot be identified with the indi- 
vidual; your identity will be protected.) 

2. Please rate your reactions on the five point scale from 1, strongly 
disagree^ to 5/ strongly agree. 



Strongly Strongly 
Disagree A^ee 



1# Parents should participate 

in various^ phases of the 1 
reading/language arts pro- 
gram, 

2. Parents should be thoroughly 
informed about the reading/ 1 
language arts program. 

3. The developmental reading/language 
arts program should be empha- 1 
sized for all pupils regard- 
less of their achievement e 

4. Most teachers practice flexible 
grouping procedures, 1 

5* Creative teaching is encouraged 
by the school administration. X 

Pupils have a positive attitude 
gp^(^ toward the reading specialists, 1 



strongly 
Disagree 

7* Remedial reading for those 1 : 2 

pupils who are considerably 
below grade level should be 
dbrongly emphasirzed. 

8. If given the opportunii^iy / 

most pupils will do a con- i' t 2 

siderable amount of reading 
on their own^ 

9. Teachers are actively involved in 

the planning of new programs . 1 : 2 

10. Most teachers are receptive 

to offers of professional 1^2 
help from supervisors and 
administrators . 

11. School administrators provide 

an atmosphere conducive to 1:2 
learning. 

ife. The school system furnishes 

adequate instructional 1:2 
materials. 

13. Pupils handle instructional materi- 
als, books and equipment with re- 
spect. 1 : 2 

14^ The school administration fully 

supports the. work, of the reading/ 1 
language arts supervisors and/or 
specialists. 

15. Contemplated changes and new 
programs are thoroughly ex- 1' 
plained to parents 5 

16. Parents feel that the* reading/ 
language arts program is reaching 
the needs of 

(1) all pupils 1 

(2) average pupils 1 

(3) pupils below grade level 1 

(4) pupils well above grade level 1 



2 : 



2 
2 
2 
2 



3 
3 
3 
3 



Strongly 
A^ree 



: 3 : 4 



3 ; 4 



3 : 4 



3 : 4 
3 : 4 



2:3:4 



4 
4 
4 

A 



5 
5 



5 



5 
5 
5 
5 



ERLC 



strongly 
Disagree 

17. Most teachers make a con- 
certed effort to individualize i : 2 
instruction. 

18. Most school leaders are know- 
ledgeable about the* reading/ i : 2 
language arts program^ 

19 « Adequate supplementary materials are 

furnished to the teachers • 1:2 

20, The administration encourages 
experimentation and innovation. 1 : 2 

21. Parents ap\)ear to be satisfied 

with the reading program. 1 : 2 



3 

Strongly 
Agree ^ 

t 3 : 4 : 5 

: 3 s 4 : 5' 

: 3 : 4 : S 
: 3 : 4 : 5 
: 3 : 4 : 5 
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Part II - Directions: 

1, Please make a check in the appropriate column to i-ndicate the fre- 
quency to which you- used the foilowi-ng- techniques or activities to 
teach reading during your past year of teaching. 





Daily- 


2-3 Times 2-3 Times 
Weekly Monthly Never 


1. 


Linguistic Materials 


> • « 






» • « 


3. 


Basal Readers 


■ • 


4. 


Individualized Readina j 


• • s 


5. 


Multi **Gi:h3 G bonk«! and/n** 
materials 




6, 


Reading to Students 


! : ! 


7. 


Recreational Reading 


! : : 


8. 


Show "n Tell 


I : : 


9. 


Student interpretation & \ 
critical response j 


1 ": : 


10, 


Reading Groups j 




11, 


Role Playing 




12. 


Creative Writing 




13. 


Recordings 




14. 


Tapes (audio) ! 




15. 


Films 




16, 


Pupil-Teacher Evaluation 




nr 


Contemporary Prose and 
Poetry 





18, Games 



rn?/-- G^oup Planning 
hKJC — - 



1 



Pc>rt II - (con't,) 






We ekl^^ 


2-3 Times Never 
Monthly 




Individual Pupil Planning : • 


• 
• 


21. 


Involvement of Parents in . 
your reading program 


• 
• 


22, 


Integration of language 

skills development with j • 
concenw areos 




: 


23. 


Field Trips \ s 


• 

L, . 


24. 


■ • 

Pupils dictating stories ; : 
uo xeacners 


• 
• 


ZD* 


iraoxcionai prose anQ • • 
Poetry 


• 
• 


26. 


Observation of other . : 
Classes o programs 




27. 


Parent-Teacher Evaluation ' . • 
con j^t^re noes 


• 
• 


Zo • 


Classroom uemonscrauions. • * 


: 

■ > ■■ . »i 1 ^^^^ 


29. 


Workbooks : : 




30. 


Community Resources • : 




31, 


Library : : 




32. 


Teacher Aides, Clerical • : 
Assistants 




33, 


Programmed Materials : : 


• 
• 


34. 


Flexible Grouping • ; 
Procedures 


• 
• 
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Part III - Directions: 

Please answer all of the following ttenw. Use the back of this Instrument 
for any additional con&ents you wish to make. 

I. List In Rank Order the ten most effective techniques and/or activities for 
teaching reading. 

1. 6. 



2. 7. 



3. 8. 



4. 9. 



5. 10. 



II. List In Rank Order your objectives for your reading program. 
I. 



2. 



3. 



4. 
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!!• Con't. 
6. 



7. 



8. 



9. 



10. 



HIo To what extent do you fe^l the activities involvecoent of parents with teachers 
enhances the effectiveness of the reading program for: 

(a) culturally deprived students 

(1) not at all (2) moderaftely (3) to a considerable degree (4) extensively 

(b) Slow learners 

(1) not at all (2) moderately (3) to a considerable degree (4) extensively 

(c) average £ .udentis 

(1) not at all (2) moderately (3) to a considerable degree (4) extensively 

(d) above average students 

(1) not at all (2) moderately (3) to a considerable degree (4) extensively* 
IV. How do you rate your effectiveness as a teacher of reading to; 

(a) culturally deprived students 

(1) Idif (Z) moderate (3) strong (4) excellent 

(b) average pupils 

(1) low (2) moderate (3) strong (4) excellent 
^ (c) above average students 

(1) low (2) moderate (3) strong (4) excellent 



8 



V» What would you consider the most effective way to Improve your competency In 
teachd^g reading? 



VI# To what extent do non-standard dialects limit a child's ability to success in 
a reading program? 

(a) students well below grade level 

(1) not at all (2) moderately (3) to a considerable degree (4) extensively 

(b) slow learners 

(1) not at all (2) moderately (3) to a considerable degree (4) extensively 

(c) average students 

(I) not at all (2) moderately (3) to a considerable degree (4) extensively 

(d) aboVe average students 

(1; nofJit all (2) moderately <3) to a considerable degree (4) extensively 
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APPENDIX D 



DIRECTORS QUESTIONNAIRE 
INFORMATION FOR COORDINATED EVALUATION 




SPECIALIZED EDUCATIONAL CONSULTANT SERVICES 
Box 3146 College Station, Durham, RC 27708 

May n, 1971 



Directors Questionnaire: Language Arts/Reading Projects 

The follov;ing requests for information are designed to assist the 
coordinated evaluation team in preparing its report to you and 
NCTE. 

A. We are interested in your perception of the impact that the 
Language Arts/Reading Project has had on your cooperating 
institution (university) . The following are items you may 
wish to include in your report: 

1. Number of professors or other university staff involved 
(type of ir ^vement)* 

2. Number of graduate students involved (type of involvement). 

3. Number of undergraduates involved (type of involvement). 

B. One charge that we had was to devote attention to the outreach, 
transfer or multiplier effect of the projects. Would you please 
address two or three paragraphs to this portion of your project. 
You may want to include tho following among some of your points: 

1. Number and types of visitors to the project. 

2. Number of outside teachers and staff brought into the project 
and their level of involvement. 

3. Number of teachers outside project group used in 
consultantships and the degree of involvement* 

4. Number of times project staff presented dimensions of 
project to audiences. 

5. other evidence of "multiplies" effect. 

C. Projects provided for different levels of parent involvement. 
Different groups of parents were involved at different levels. 
Devote two or three paragraphs to a description of this phase 
of your project. You may want to include the following: 

1. Number of parents. 

2. Evidence of parent participation. 

3. Were there regular meetings. Describe. 

4. What were some roles parents played in your project. 

ERLC 




SP ECIALIZED EDUCATIONAL CONSULTANT SERVICES 
Box 6145 College Station, Durham, RC 27708 

May 17, 1971 



TO: NCTE/OE Language Arts/Reading Project Evaluators 

FROM: Coordinated Evaluation Team 

Drs. Ray, Pittillo, Katzenmeyer and Peck 

SUBJECT: Information for Coordinated Evaluation 



At our Berkeley meeting sometime in the late and sleepy hours of 
our final .session Bob Ruddell asked that we prepare a "reminder 
list" of those materials that should be or have been sent to us 
in order for the coordinated evaluation to be completed. 

Therefore we submit the following list: 

1. Pre-Test on Teacher Attitude and Practices Survey 
(we have these) 

2. Post-Test on Teacher Attitude and Practices Survey 
(mailed to you on May 17, 1971) 

3. Project Directors Survey (see enclosed) 

4. Pre Data Cards on Project Teachers (see previous memo for 
card formats) 

5* Post Data Cards on Project Teachers (see previous memo 
for card formats) 

6. Pre Data Cards on Project Students (see previous memo for 
card format) 

7. Post Data Cards on Project Students (see previous memo for 
card format) 



cc: Robert Hogan 

Doris Gunderson 

Ends • 
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OHIO STATE PUPIL PERFORMANCE RESULTS 
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